LiveKit Docker Deployment Notes/Issues

livekit-docker-oops.md

While I'm normally not a docker guy; I'm part of a large project in which the deployment will matter...so I figured I better start building things in Docker. This project will use livekit, so I decided to deploy that. Oh boy. There are some problems that the documentation didn't clearly cover. So...here's my janky set of notes:

All References To docker-compose should be docker compose.

They did not update the deployment script; therefore it generates non-working initializations. They are using old outdated information that is not compatible with the new versions of docker the script installs. Yes...it installs versions incompatible with itself. The only issue really is that all references to docker-compose should be docker compose. Both in the documentation and in the script.

No Port 80 When The Docs Say You Need Port 80

The documentation will clearly tell you that you need port 80 forwarded to your livekit:

• 80 - for TLS issuance

The only problem with this is there's no port 80 in the docker compose for any of the machines. None. Port 80 is not on any of the machines. None. I cannot see why you need to send port 80 to a bunch of docker VMs that don't have port 80 open. I try to hit my livekit over the lan on port 80 and it's not open.

But if there's no 80 how is it getting letsencrypt certs? It's not using http-01 challenge. Oh...it tries to. You will see it in the logs as failing. What it's supposed to use is TLS-ALPN-01, which happens over 443. This is something that probably can be fixed in a proper config/docker compose; however it will try the TLS method after a while.

Notes About Proxy

I have a stupid funky network setup in the lab where I'm deploying this. Single IPv4 and a whole /56 of v6. My lab runs dual-stack and even worse....all my v6 traffic just goes direct to the box. That's literally been fine for most stuff since I have a single nginx. However the livekit requires extra work since it uses HTTPS and TLS over 443. Primarily...I had to enable stream plugins for nginx.

nginx.conf:

stream {

    map $ssl_preread_server_name $backend {
        livekit.yourdomain.com 192.168.1.11:443;
        turn.yourdomain.com 192.168.1.11:443;
        default 127.0.0.1:8443;
    }

    server {
        listen 443;
        proxy_pass $backend;
        ssl_preread on;
    }
}

The AAAA records for those domains go directly to the unicast v6 of that machine, bypassing nginx entirely. This does mean you have to move all of your ssl sites to a different port since the stream listener will take over 443. Not really a big deal, just have to change a bunch of listen ports to 8443 and reload.

As about as far as I've gotten is getting certs issued; which means at least proxying and talking to the server works. Lots of complaints in the logs about things they didn't configure though.

But at least this should help anyone trying to deploy the thing.