Nomad+Consul TLS Story

nomad-consul-tls.md

Nomad+Consul TLS Story

See https://github.com/schmichael/nomad-tls for sample configs.

Nomad issues

• Consul uses ssl instead of tls config parameters
• Consul uses verify_ssl instead of verify_tls
• consul.cert_file should specify that it's the client certificate used for mutual authentication.
  • consul.{ca,cert,key}_file should default to tls.{ca,cert,key}_file
• verify_ssl docs should specify that the consul.address must include a host/IP -- this seems like a bug as everywhere else we validate that the certificate is valid for {client,server}.{datacenter,region}.{consul,nomad}
• There should be a facility to configure Nomad's CLI to use TLS by default (with the proper CA)
  • Environment variable?
  • Config file(s)? (eg /etc/nomad/*.hcl overridden by $HOME/.nomad/*.hcl?)

Consul issues

Consul docs

All servers must have a certificate valid for server.. or the client will reject the handshake.

I couldn't find a definition of <domain>. It seems to mean nomad or consul. However traditionally certificates are issued to "domains" in the DNS sense of the word, so it's quite confusing.

Why use TLS w/Consul?

Similar to the <domain> issue, the reason to use TLS to communicate from Nomad to Consul is a bit non-intuitive.

The first benefit people usually associate with TLS is encryption across the wire. However Nomad communicates with Consul locally (the integration doesn't even work entirely with a shared Consul instance) -- you don't need TLS to communicate securely with localhost.

The reason why TLS is still beneficial locally is to ensure what Nomad is talking to is actually Consul. Any unprivileged process can bind to 127.0.0.1:8500 and impresonate Consul. Using TLS means in order to impersonate Consul the process has access to Consul's certificate+key which (should) require privileged access.