Skip to content

Instantly share code, notes, and snippets.

@binarytemple

binarytemple/connect.to.docker.machine.docker.service.using.local.certs.markdown

Last active April 7, 2016 22:35
Show Gist options
  • Save binarytemple/4cbcf8e4917d11b356106854f5ffebc1 to your computer and use it in GitHub Desktop.
Save binarytemple/4cbcf8e4917d11b356106854f5ffebc1 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
connect.to.docker.machine.docker.service.using.local.certs.markdown

Today I'm sharing some notes and experiments I've made while investigating the integration of Elixir and Docker. For added fun I'm going to be testing/developing on OSX using docker instances managed by docker-machine. As usual we will also have the wierd glitchy versions of standard tools distributed by OSX to deal with. Lets get started.

The openssl bundled with OSX ( OpenSSL 0.9.8zg 14 July 2015 ) was unable to successfully talk to the modern SSL shipped with docker machine. Not the first time I've bumped up against problems with the stock OSX openssl, but I'm guessing unless it's got an Apple log.. that's unlikely to change any time soon. I then switched to using an openssl distributed via homebrew ( OpenSSL 1.0.2g 1 Mar 2016 ) and it worked perfectly well.

/usr/local/Cellar/openssl/1.0.2g/bin/openssl s_client -prexit \
-connect 192.168.103.138:2376 \
-CAfile /Users/adminuser/.docker/machine/certs/cert.pem \
-key /Users/adminuser/.docker/machine/certs/key.pem \
-CAfile /Users/adminuser/.docker/machine/certs/ca.pem
depth=1 O = adminuser
verify return:1
depth=0 O = adminuser.local
verify return:1
140735136731216:error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate:s3_pkt.c:1472:SSL alert number 42
140735136731216:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177:
CONNECTED(00000003)
---
Certificate chain
 0 s:/O=adminuser.local
   i:/O=adminuser
---
Server certificate
-----BEGIN CERTIFICATE-----
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-----END CERTIFICATE-----
subject=/O=adminuser.local
issuer=/O=adminuser
---
Acceptable client certificate CA names
/O=adminuser
Client Certificate Types: RSA sign, ECDSA sign
Requested Signature Algorithms: RSA+SHA256:ECDSA+SHA256:RSA+SHA384:ECDSA+SHA384:RSA+SHA1:ECDSA+SHA1
Shared Requested Signature Algorithms: RSA+SHA256:ECDSA+SHA256:RSA+SHA384:ECDSA+SHA384:RSA+SHA1:ECDSA+SHA1
Peer signing digest: SHA384
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 1271 bytes and written 138 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: 
    Session-ID-ctx: 
    Master-Key: CA75210B7CD6D8A42476767F20C9E60ADFEF35E19BE67AC275FFF869CD8D8524E0C446F7248BF973A39A739754713106
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1460050852
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
---
Certificate chain
 0 s:/O=adminuser.local
   i:/O=adminuser
---
Server certificate
-----BEGIN CERTIFICATE-----
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-----END CERTIFICATE-----
subject=/O=adminuser.local
issuer=/O=adminuser
---
Acceptable client certificate CA names
/O=adminuser
Client Certificate Types: RSA sign, ECDSA sign
Requested Signature Algorithms: RSA+SHA256:ECDSA+SHA256:RSA+SHA384:ECDSA+SHA384:RSA+SHA1:ECDSA+SHA1
Shared Requested Signature Algorithms: RSA+SHA256:ECDSA+SHA256:RSA+SHA384:ECDSA+SHA384:RSA+SHA1:ECDSA+SHA1
Peer signing digest: SHA384
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 1271 bytes and written 138 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: 
    Session-ID-ctx: 
    Master-Key: CA75210B7CD6D8A42476767F20C9E60ADFEF35E19BE67AC275FFF869CD8D8524E0C446F7248BF973A39A739754713106
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1460050852
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

Wrapping that up in a script we find ourselves with the following util/docker-machine-test-connection :

INSTANCE=${1:- you must provide the docker-machine instance name as the argument to this script}

CMD="$(find /usr/local/Cellar/openssl -maxdepth 1 -mindepth 1 | sort -n | head -n1 )/bin/openssl"

eval $(docker-machine env ${INSTANCE})

export DOCKER_HOST=$(echo $DOCKER_HOST| sed 's_tcp://__')

$CMD s_client -prexit -connect $DOCKER_HOST -CAfile /Users/${USER}/.docker/machine/certs/cert.pem -key /Users/${USER}/.docker/machine/certs/key.pem -CAfile /Users/${USER}/.docker/machine/certs/ca.pem

And, use a similar script to generate our stunnel.conf configuration files for the stunnel SSL proxy.

Lets examine the file util/generate-docker-machine-stunnel_conf:

INSTANCE=${1:- you must provide the docker-machine instance name as the argument to this script}
CMD="$(find /usr/local/Cellar/openssl -maxdepth 1 -mindepth 1 | sort -n | head -n1 )/bin/openssl"
eval $(docker-machine env ${INSTANCE})
export DOCKER_HOST=$(echo $DOCKER_HOST| sed 's_tcp://__')

cat <<END > "$INSTANCE-stunnel.conf"
 
client = yes
debug                  = daemon.info
foreground             = yes 
log                    = append
output                 = /tmp/stunnel.log

[docker]
accept = 8888
connect = $DOCKER_HOST
cert = /etc/stunnel/stunnel.pem
key=/Users/${USER}/.docker/machine/certs/key.pem 
CAfile=/Users/${USER}/.docker/machine/certs/ca.pem
cert = /Users/${USER}/.docker/machine/certs/cert.pem 
END

Lets fire up stunnel using that configuration file:

 % stunnel local-stunnel.conf                                                                        (git)-[master]-
2016.04.07 23:18:08 LOG5[ui]: stunnel 5.31 on x86_64-apple-darwin15.3.0 platform
2016.04.07 23:18:08 LOG5[ui]: Compiled/running with OpenSSL 1.0.2g  1 Mar 2016
2016.04.07 23:18:08 LOG5[ui]: Threading:PTHREAD Sockets:POLL,IPv6 TLS:ENGINE,FIPS,OCSP,PSK,SNI
2016.04.07 23:18:08 LOG5[ui]: Reading configuration from file /common/erldocker/local-stunnel.conf
2016.04.07 23:18:08 LOG5[ui]: UTF-8 byte order mark not detected
2016.04.07 23:18:08 LOG5[ui]: FIPS mode disabled
2016.04.07 23:18:08 LOG6[ui]: Initializing service [docker]
2016.04.07 23:18:08 LOG6[ui]: Loading certificate from file: /Users/bryanhunt/.docker/machine/certs/cert.pem
2016.04.07 23:18:08 LOG6[ui]: Certificate loaded from file: /Users/bryanhunt/.docker/machine/certs/cert.pem
2016.04.07 23:18:08 LOG6[ui]: Loading private key from file: /Users/bryanhunt/.docker/machine/certs/key.pem
2016.04.07 23:18:08 LOG6[ui]: Private key loaded from file: /Users/bryanhunt/.docker/machine/certs/key.pem
2016.04.07 23:18:08 LOG4[ui]: Service [docker] needs authentication to prevent MITM attacks
2016.04.07 23:18:08 LOG5[ui]: Configuration successful

And test using curl:

curl 127.0.0.1:8888/images/json | jsonpipe

/       []
/0      {}
/0/Id   "sha256:1a56577afb682e29ad99477fe70f66ba948892a28a4a46b27e6c32eb17c6621e"
/0/ParentId     "sha256:199a898e5db05408af74801dfd959e39d924e981b26ed2382f4947918e02aadf"
/0/RepoTags     []
/0/RepoTags/0   "binarytemple/tcpreplay:latest"
/0/RepoDigests  null
/0/Created      1459453888
/0/Size 542836017
/0/VirtualSize  542836017
/0/Labels       {}
/1      {}
/1/Id   "sha256:f813ec5e0d8fda69fbf5a37ade2014fa72a421f0e378700c6e48751d85980111"
/1/ParentId     ""
/1/RepoTags     []
/1/RepoTags/0   "elasticsearch:1.5.2"
/1/RepoDigests  null
/1/Created      1455761869
/1/Size 344337046
/1/VirtualSize  344337046
/1/Labels       {}
/2      {}

Looking good. Next step I'd like to investigate what's involved in getting knewters fork of erldocker talking to my docker instances. Something else of interest would be elixir-socket - it has nice abstractions for SSL connections.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment