[heroku: Heorku x WebSocket] Configurations for deploying to heroku. #heroku

heroku_websocket_note.md

TIPS

• 最近 heroku redis v6 になって SSL: CERTIFICATE_VERIFY_FAILED が出るようになった
  • https://help.heroku.com/HC0F8CUS/redis-connection-issues
• 古いチャネルを破棄するときのConnectionClosedError ＃135
• Heroku ドメイン my-app.herokuapp.com を利用する場合は WebSocket 内部のポーリングが途切れる可能性がある
  • WebSocket には通信経路に HTTP / HTTPS が混在するとき、接続が途切れたコネクションを検知できなかったり、内部ポーリング ( Ping Pong ) が途切れたりするバグが存在する
  • このため TLS/SSL 証明書を購入し、WebSocket サーバである Uvicorn に渡せない場合 WebSocket による通信の全ての経路が TLS/SSL にならず、結果的に「なんか途中で途切れた」みたいなことが起きる
  • 再接続 ( リロードするなり ) すればよいが、商用サービスなどではかならずこの問題をつぶす必要がある

COMPOSITION

• Nginx
• Django x Django Channels
  • WSGI: uWSGI
  • ASGI: Uvicorn

CONTAINERS

Dockerfile.web

# Dockerfile.web for Heroku deployment.
#
FROM python:3.7
ENV  PYTHONUNBUFFERED 1

# Set up project root.
RUN     mkdir app
WORKDIR /app

# Setup nginx.
ARG  DOMAIN
RUN  apt-get update -qq && apt-get install -y build-essential libpq-dev nodejs
RUN  wget "http://nginx.org/keys/nginx_signing.key" && \
     apt-key add nginx_signing.key && \
     echo "deb http://nginx.org/packages/debian/ stretch nginx" >> /etc/apt/sources.list && \
     echo "deb-src http://nginx.org/packages/debian/ stretch nginx" >> /etc/apt/sources.list && \
     apt-get update
RUN  apt-get install -y nginx && \
     rm -rf /etc/nginx/conf.d/* && \
     rm -rf /etc/nginx/sites-enabled/*
COPY ./.docker/web/uwsgi_params /etc/nginx/uwsgi_params
COPY ./.docker/web/heroku.conf  /etc/nginx/conf.d/default.conf

# Install packages before copy sources, for building cache.
COPY ./.docker/app/requirements.txt /app/.docker/app/requirements.txt
RUN  pip install -r /app/.docker/app/requirements.txt

# Copy source codes.
COPY . /app

# Run application server.
CMD sed -e "s/PORT/${PORT}/g" -i /etc/nginx/conf.d/default.conf && \
    sed -e "s/DOMAIN/${DOMAIN}/g" -i /etc/nginx/conf.d/default.conf && \
    nginx -c /etc/nginx/nginx.conf && \
    uwsgi --ini /app/.docker/app/django.ini --master -d /var/log/uwsgi.log && \
    uvicorn djangdock.asgi:application --host 0.0.0.0 --port 8002 --proxy-headers

Nginx conf

server_tokens off;

# The upstream component nginx needs to connect to.
upstream django {
  ip_hash;
  server localhost:8001;
}

server {
  listen      PORT;
  charset     utf-8;
  server_name DOMAIN;

  access_log /dev/stdout;
  error_log  /dev/stderr debug;

  client_max_body_size 100M;

  # proxy_set_header Origin $http_origin;
  proxy_set_header Origin $scheme://$host;
  proxy_set_header Host $http_host;
  proxy_set_header X-CSRF-Token $http_x_csrf_token;
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header X-Forwarded-Proto $scheme;
  proxy_request_buffering off;
  proxy_redirect off;
  proxy_ssl_server_name on;

  location /static {
    alias /app/static;

    # this rewrites all the requests to the maintenance.html
    # page if it exists in the doc root. This is for capistrano's
    # disable web task
    if (-f $document_root/maintenance.html) {
      rewrite  ^(.*)$  /maintenance.html last;
      break;
    }

    # If the file exists as a static file serve it directly without
    # running all the other rewrite tests on it
    if (-f $request_filename) {
      break;
    }

    # check for index.html for directory index
    # if it's there on the filesystem then rewrite
    # the url to add /index.html to the end of it
    # and then break to send it to the next config rules.
    if (-f $request_filename/index.html) {
      rewrite (.*) $1/index.html break;
    }

    # this is the meat of the rack page caching config
    # it adds .html to the end of the url and then checks
    # the filesystem for that file. If it exists, then we
    # rewrite the url to have explicit .html on the end
    # and then send it on its way to the next config rule.
    # if there is no file on the fs then it sets all the
    # necessary headers and proxies to our upstream pumas
    if (-f $request_filename.html) {
      rewrite (.*) $1.html break;
    }
  }

  # Proxy to Django WebSocket server uvicorn.
  location /ws/ {
    proxy_pass http://localhost:8002;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
  }

  # Finally, send all non-media requests to the Django server.
  location / {
    uwsgi_pass  django;
    include     /etc/nginx/uwsgi_params;  # the uwsgi_params file you installed
  }
}

Dockerfile.worker

# Dockerfile.worker for Heroku deployment.
#
FROM python:3.7
ENV  PYTHONUNBUFFERED 1

# Set up project root.
RUN     mkdir app
WORKDIR /app

# Install packages before copy sources, for building cache.
COPY ./.docker/app/requirements.txt /app/.docker/app/requirements.txt
RUN  pip install -r /app/.docker/app/requirements.txt

# Copy source codes.
COPY . /app

# Run worker process.
RUN mkdir -p /root/run/celery && rm -rf /root/run/celery/*
CMD celery -A djangdock worker --pidfile="/root/run/celery/%n.pid" --loglevel=info

DEPLOYMENT

$ heroku container:push --recursive -a my-app --arg DOMAIN=my-app.herokuapp.com
$ heroku container:release web worker -a my-app

# Worker Dyno をスケールアップして 1 台起動させる ( 初回のみ必要 )
$ heroku ps:scale worker=1 -a my-app