Example for Docker Swarm, Let's Encrypt and Nginx setup with no Nginx down time (answer to https://twitter.com/developius/status/892470102632923136)

0.Notes.md

This is an answer to https://twitter.com/developius/status/892470102632923136 about his SSL with Docker Swarm, Let's Encrypt and Nginx blog post and a way to not kill Nginx for certificate generation/renewal.

Assumptions

(from what I read in the blog post)

• Docker hosts have a /etc/letsencrypt directory so that certificates are on the host and not on the container.
• Docker hosts have a /var/lib/letsencrypt shared copy of the and a (docker run certbot could also re-use containers' /var/lib/letsencrypt volumes.

My proposition

I use /var/lib/letsencrypt/webroot/ as a place for the certbot Webroot plugin (--webroot) to store ACME Challenges, which are then served by Nginx server for "/.well-known/acme-challenge/xxxxxxxxx" HTTP requests only.

Because, certbot's Webroot plugin only need an HTTP (not HTTPS) server to chat with about ACME Challenges, for new domains, I first start Nginx with only the HTTP (port 80) server: the HTTPS (port 443) server block is commented, then run the certbot certonly --webroot command. Once first certificate has been issued, I can enable the HTTPS (port 443) server and reload Nginx.

Future certificate renewal (certbot renew --webroot) part is untouched (same as blog post).

1.setup.sh

# Create a directory where certbot will place ACME Challenges
# Our Nginx server will also use this as a root for serving files
# when requested for "/.well-known/acme-challenge/xxxxxxxxx".
mkdir -p /var/lib/letsencrypt/webroot

2.nginx.conf

user nginx;  
worker_processes 1;

error_log /var/log/nginx/error.log warn;  
pid /var/run/nginx.pid;


events {  
  worker_connections 1024;
}


http {  
  include /etc/nginx/mime.types;
  default_type application/octet-stream;

  log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for"';

  access_log /dev/stdout main;
  sendfile on;
  keepalive_timeout 65;

  server {
    listen 80;
    server_name _;

    # redirect from http to https
    location / {
        return 301 https://$host$request_uri;
    }

    # Rule for legitimate ACME Challenge requests (like /.well-known/acme-challenge/xxxxxxxxx)
    location ^~ /.well-known/acme-challenge/ {
        # No HTTP authentication
        allow all;
    
        # Set correct content type. According to this:
        # https://community.letsencrypt.org/t/using-the-webroot-domain-verification-method/1445/29
        # Current specification requires "text/plain" or no content header at all.
        # It seems that "text/plain" is a safe option.
        default_type "text/plain";
    
        # Change document root: this path will be given to certbot as the 
        # `-w` param of the webroot plugin.
        root /var/lib/letsencrypt/webroot;
    }

    # Hide /acme-challenge subdirectory and return 404 on all requests.
    # It is somewhat more secure than letting Nginx return 403.
    # Ending slash is important!
    location = /.well-known/acme-challenge/ {
        return 404;
    }

    access_log off;
  }

   #include /etc/nginx/nginx-https_server.conf;
}

3.nginx-https_server.conf

server {
  listen              443 ssl;
  server_name         example.com;
  ssl_certificate     /etc/letsencrypt/live/example.com/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
  ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;
  ssl_ciphers         HIGH:!aNULL:!MD5;

  location / {
     # do your thing
  }
  
  # No acme ACME Challenge stuff here: Let's Encrypt API uses HTTP for validation
}

4.docker-compose.yml

version: '3.2'

services:  
  nginx:
    image: nginx:stable-alpine
    volumes:
      - /etc/letsencrypt:/etc/letsencrypt
      - /usr/share/nginx/html:/usr/share/nginx/html
      - /var/lib/letsencrypt:/var/lib/letsencrypt
      - ${PWD}/nginx.conf:/etc/nginx/nginx.conf
      - ${PWD}/nginx-https_server.conf:/etc/nginx/nginx-https_server.conf
    deploy:
      mode: global
      placement:
        constraints:
          - node.role == manager
    ports:
      - 80:80
      - 443:443

5.generate.sh

# First time certificate generation
# (No need for 80 nor 443 ports)
docker run --rm \  
  --name letsencrypt \
  -v "/etc/letsencrypt:/etc/letsencrypt" \
  -v "/var/lib/letsencrypt:/var/lib/letsencrypt" \
  certbot/certbot certonly -n \
  -m "YOUR_EMAIL" \
   --agree-tos \
  -d example.com \
  --webroot --webroot-path /var/lib/letsencrypt/webroot/

# Now enable/create the HTTPS nginx VHost
docker exec nginx sed -r 's,^(\s*)#(include /etc/nginx/nginx-https_server.conf;)$,\1\2,' /etc/nginx/nginx.conf

# Reload Nginx
docker kill -s HUP nginx

6.renew.sh

# Command to run periodically
# (Removed /usr/share/nginx/html, superseded by /var/lib/letsencrypt/webroot)
docker run --rm \  
    --name letsencrypt \
    -v "/etc/letsencrypt:/etc/letsencrypt" \
    -v "/var/lib/letsencrypt:/var/lib/letsencrypt" \
    certbot/certbot:latest \
    renew --quiet --no-self-upgrade

# Reload Nginx
docker kill -s HUP nginx

# Improvement: only reload Nginx if renewall occured (using `--renew-hook` with an `echo` and then grepping the output in Docker host).

how to add load balancing using upstream ?