NEW VERSION OF THIS DOC https://github.com/navidshaikh/testing-adb
Bex has done a better job in building up the context and outlining the idea [here] (https://gist.github.com/bexelbie/35f421ee247c7621c0d0) . Goal here is to configure a TLS enabled Docker docker daemon running over TCP from inside CDK and configure client at Eclipse to be able to connect to it.
In order to get configure CDK + Eclipse
dockerdaemon inside CDK has to run over TCP (as well, if running on local unix socket) - This need required certs to be generated inside CDK and updating thedockerdaemon config and restarting service.- Required client side certs has to be copied over to host (from guest) for clients to connect
- Port at which
dockerinside CDK running need to be mapped at host machine port for client to easily connect
Step -1 above is taken care by incorporating steps to generate certs and updating config in Vagrantfile
Step -3 is again taken care by specifying port mapping in Vagrantfile with auto_correct option incase of port collision at host - There are few complexities (to be covered later) for libvirt provider however for Virutalbox it works as expected.
For solving problem stated in Step-2 there are following two options
- Use ssh and scp to copy over the certs from inside the box to host
- Need for Windows platform:
sshandscpavailability on host - vagrant-triggers plugin for triggering the
scpoperation as box is "provisioned"- Knowledge about the mapped
sshport at host machine for thescpcommand (working on making this automated)
- Knowledge about the mapped
- Setup VirtualBox guest additions inside guest and use vagrant synced folders feature. this will need
- a line in Vagrantfile for specifying the folder to be synced
- for atomic guest - vbox guest additions configured in a (SPC) container
- for non atomic guest - vbox guest additions configured
Note: plugin [vagrant-vbguest] (https://github.com/dotless-de/vagrant-vbguest) is another option which can set vbox guest additions for community non-atomic platforms (CentOS and Fedora)
##Hands-on NEW VERSION OF THIS DOC https://github.com/navidshaikh/testing-adb ###1. CDK + Eclipse with vbox guest additions for VirtualBox provider
vagrant plugin install vagrant-vbguest- Use following Vagrantfile
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure(2) do |config|
config.vm.box = "atomicapp/dev"
config.vm.network "forwarded_port", guest: 2376, host: 2379, auto_correct: true
if !Vagrant.has_plugin?("vagrant-vbguest")
puts "'vagrant-vbguest' plugin is required"
puts "This can be installed by running:"
puts
puts " vagrant plugin install vagrant-vbguest"
puts
exit
end
config.vm.provision "shell", inline: <<-SHELL
# =========================================================
# Generate Certs for running TLS enabled docker daemon
# Generate Certificates to use with the docker daemon
# Instructions sourced from http://docs.docker.com/articles/https/
# Get the certificate location, i.e. setting the DOCKER_CERT_PATH variable
. /etc/sysconfig/docker
# randomString from http://utdream.org/post.cfm/bash-generate-a-random-string
# modified to echo value
function randomString {
# if a param was passed, it's the length of the string we want
if [[ -n $1 ]] && [[ "$1" -lt 20 ]]; then
local myStrLength=$1;
else
# otherwise set to default
local myStrLength=8;
fi
local mySeedNumber=$$`date +%N`; # seed will be the pid + nanoseconds
local myRandomString=$( echo $mySeedNumber | md5sum | md5sum );
# create our actual random string
#myRandomResult="${myRandomString:2:myStrLength}"
echo "${myRandomString:2:myStrLength}"
}
# Get a temporary workspace
dir=`mktemp -d`
cd $dir
# Get a random password for the CA and save it
passfile=tmp.pass
password=$(randomString 10)
echo $password > $passfile
# Generate the CA
openssl genrsa -aes256 -passout file:$passfile -out ca-key.pem 2048
openssl req -new -x509 -passin file:$passfile -days 365 -key ca-key.pem -sha256 -out ca.pem -subj "/C=/ST=/L=/O=/OU=/CN=example.com"
# Generate Server Key and Sign it
openssl genrsa -out server-key.pem 2048
openssl req -subj "/CN=example.com" -new -key server-key.pem -out server.csr
# Allow from 127.0.0.1
extipfile=extfile.cnf
echo subjectAltName = IP:127.0.0.1 > $extipfile
openssl x509 -req -days 365 -in server.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out server-cert.pem -passin file:$passfile -extfile $extipfile
# Generate the Client Key and Sign it
openssl genrsa -out key.pem 2048
openssl req -subj '/CN=client' -new -key key.pem -out client.csr
extfile=tmp.ext
echo extendedKeyUsage = clientAuth > $extfile
openssl x509 -req -days 365 -in client.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out cert.pem -extfile $extfile -passin file:$passfile
# Clean up
# set the cert path as configured in /etc/sysconfig/docker
## Move files into place
mv ca.pem $DOCKER_CERT_PATH
mv server-cert.pem $DOCKER_CERT_PATH
mv server-key.pem $DOCKER_CERT_PATH
# since the default user is vagrant and it can run docker without sudo
CLIENT_SIDE_CERT_PATH=/home/vagrant/.docker
mkdir -p $CLIENT_SIDE_CERT_PATH
cp $DOCKER_CERT_PATH/ca.pem $CLIENT_SIDE_CERT_PATH
mv cert.pem key.pem $CLIENT_SIDE_CERT_PATH
chown vagrant:vagrant $CLIENT_SIDE_CERT_PATH
chmod 0444 $CLIENT_SIDE_CERT_PATH/ca.pem
chmod 0444 $CLIENT_SIDE_CERT_PATH/cert.pem
chmod 0444 $CLIENT_SIDE_CERT_PATH/key.pem
chown vagrant:vagrant $CLIENT_SIDE_CERT_PATH/ca.pem
chown vagrant:vagrant $CLIENT_SIDE_CERT_PATH/cert.pem
chown vagrant:vagrant $CLIENT_SIDE_CERT_PATH/key.pem
chmod -v 0400 $DOCKER_CERT_PATH/ca.pem $DOCKER_CERT_PATH/server-cert.pem $DOCKER_CERT_PATH/server-key.pem
## Remove remaining files
cd
echo rm -rf $dir
# ============= end of script for generating the certs for TLS enabled docker daemon===
sed -i.back '/OPTIONS=*/c\OPTIONS="--selinux-enabled -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --tlscacert=/etc/docker/ca.pem --tlscert=/etc/docker/server-cert.pem --tlskey=/etc/docker/server-key.pem --tlsverify"' /etc/sysconfig/docker
sudo systemctl restart docker
SHELL
# setup the synced folders
config.vm.synced_folder ".", "/home/vagrant/.docker/"
config.vbguest.no_remote = true
end-
Run
vagrant upthis will install the vbox guest additions inside the guest, (be patient) -
you should have the certs in your current directory
-
clients can make connection using
127.0.0.1:2379and using the certs present in local directory -
for testing with docker do (on host)
cp -r *.pem ~/.docker docker -H 127.0.0.1:2379 --tlsverify images
###2. CDK + Eclipse with scp and ssh for VirtualBox provider NEW VERSION OF THIS DOC https://github.com/navidshaikh/testing-adb
vagrant plugin install vagrant-triggers- Export following into a
Vagrantfile
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure(2) do |config|
config.vm.box = "atomicapp/dev"
config.vm.network "forwarded_port", guest: 2376, host: 2379, auto_correct: true
if !Vagrant.has_plugin?("vagrant-triggers")
puts "'vagrant-triggers' plugin is required"
puts "This can be installed by running:"
puts
puts " vagrant plugin install vagrant-triggers"
puts
exit
end
config.vm.provision "shell", inline: <<-SHELL
# =========================================================
# Generate Certs for running TLS enabled docker daemon
#!/bin/bash
# Todo: move the files into place
# Generate Certificates to use with the docker daemon
# Instructions sourced from http://docs.docker.com/articles/https/
# Get the certificate location, i.e. setting the DOCKER_CERT_PATH variable
. /etc/sysconfig/docker
# randomString from http://utdream.org/post.cfm/bash-generate-a-random-string
# modified to echo value
function randomString {
# if a param was passed, it's the length of the string we want
if [[ -n $1 ]] && [[ "$1" -lt 20 ]]; then
local myStrLength=$1;
else
# otherwise set to default
local myStrLength=8;
fi
local mySeedNumber=$$`date +%N`; # seed will be the pid + nanoseconds
local myRandomString=$( echo $mySeedNumber | md5sum | md5sum );
# create our actual random string
#myRandomResult="${myRandomString:2:myStrLength}"
echo "${myRandomString:2:myStrLength}"
}
# Get a temporary workspace
dir=`mktemp -d`
cd $dir
# Get a random password for the CA and save it
passfile=tmp.pass
password=$(randomString 10)
echo $password > $passfile
# Generate the CA
openssl genrsa -aes256 -passout file:$passfile -out ca-key.pem 2048
openssl req -new -x509 -passin file:$passfile -days 365 -key ca-key.pem -sha256 -out ca.pem -subj "/C=/ST=/L=/O=/OU=/CN=example.com"
# Generate Server Key and Sign it
openssl genrsa -out server-key.pem 2048
openssl req -subj "/CN=example.com" -new -key server-key.pem -out server.csr
# Allow from 127.0.0.1
extipfile=extfile.cnf
echo subjectAltName = IP:127.0.0.1 > $extipfile
openssl x509 -req -days 365 -in server.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out server-cert.pem -passin file:$passfile -extfile $extipfile
# Generate the Client Key and Sign it
openssl genrsa -out key.pem 2048
openssl req -subj '/CN=client' -new -key key.pem -out client.csr
extfile=tmp.ext
echo extendedKeyUsage = clientAuth > $extfile
openssl x509 -req -days 365 -in client.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out cert.pem -extfile $extfile -passin file:$passfile
# Clean up
# set the cert path as configured in /etc/sysconfig/docker
## Move files into place
mv ca.pem $DOCKER_CERT_PATH
mv server-cert.pem $DOCKER_CERT_PATH
mv server-key.pem $DOCKER_CERT_PATH
# since the default user is vagrant and it can run docker without sudo
CLIENT_SIDE_CERT_PATH=/home/vagrant/.docker
mkdir -p $CLIENT_SIDE_CERT_PATH
cp $DOCKER_CERT_PATH/ca.pem $CLIENT_SIDE_CERT_PATH
mv cert.pem key.pem $CLIENT_SIDE_CERT_PATH
chown vagrant:vagrant $CLIENT_SIDE_CERT_PATH
chmod 0444 $CLIENT_SIDE_CERT_PATH/ca.pem
chmod 0444 $CLIENT_SIDE_CERT_PATH/cert.pem
chmod 0444 $CLIENT_SIDE_CERT_PATH/key.pem
chown vagrant:vagrant $CLIENT_SIDE_CERT_PATH/ca.pem
chown vagrant:vagrant $CLIENT_SIDE_CERT_PATH/cert.pem
chown vagrant:vagrant $CLIENT_SIDE_CERT_PATH/key.pem
chmod -v 0400 $DOCKER_CERT_PATH/ca.pem $DOCKER_CERT_PATH/server-cert.pem $DOCKER_CERT_PATH/server-key.pem
## Remove remaining files
cd
echo rm -rf $dir
# ============= end of script for generating the certs for TLS enabled docker daemon===
sed -i.back '/OPTIONS=*/c\OPTIONS="--selinux-enabled -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --tlscacert=/etc/docker/ca.pem --tlscert=/etc/docker/server-cert.pem --tlskey=/etc/docker/server-key.pem --tlsverify"' /etc/sysconfig/docker
sudo systemctl restart docker
SHELL
# triggers
config.trigger.after :provision do
hport = `vagrant ssh-config`.split("\n ").find{|e| e.start_with?("Port") }.split(" ")[1]
run "scp -r -P #{hport} -o StrictHostKeyChecking=no -i .vagrant/machines/default/virtualbox/private_key [email protected]:/home/vagrant/.docker/* ."
end
end-
Run
vagrant up --provider virtualbox(in the same directory where Vagrantfile exist) -
Run
vagrant provision -
You should have the certs in your current directory which is created (or updated) as part of above step
-
To test the client connection to
dockerdaemon inside CDK: In the Vagrantfile at line no: 7, we have mapped host port2379todockerdaemon port (2376) inside CDK, which means that you can access the daemon at (host machine) 127.0.0.1:2379. Following is an example of connecting viadockerCLI to daemon over TLS enabled TCP connection# copy the certs (generated above) to proper place where `docker` (by default) look up cp -r *.pem ~/.docker docker -H 127.0.0.1:2379 --tlsverify images
Note: On host machine, if you have port 2379 in use, vagrant will try to auto_correct it and assign a different port. You need to keep an eye on the corrected port while vagrant is brigning the machine up. Note that this vagrant
auto_correctfeature works as expected for Virtualbox provider but for libvirt provider it does not auto_correct and does not even fail! -
To test out the connection with Eclipse kindly check https://www.eclipse.org/community/eclipse_newsletter/2015/june/article3.php and this Video by Xavier Coulon https://www.youtube.com/watch?v=RUgEgtLux8Q. More Eclipse Docker Tooling documentations are at <https://wiki.eclipse.org/Linux_Tools_Project/Docker_Tooling/User_Guide>
I still can't get OSX docker cli to connect to this. Any info on how would be appreciated.
I think the only reason eclipse works is that it does not seem to check certificates which is another issue.