NEW VERSION OF THIS DOC https://github.com/navidshaikh/testing-adb
Bex has done a better job in building up the context and outlining the idea [here] (https://gist.github.com/bexelbie/35f421ee247c7621c0d0) . Goal here is to configure a TLS enabled Docker docker daemon running over TCP from inside CDK and configure client at Eclipse to be able to connect to it.
In order to get configure CDK + Eclipse
dockerdaemon inside CDK has to run over TCP (as well, if running on local unix socket) - This need required certs to be generated inside CDK and updating thedockerdaemon config and restarting service.- Required client side certs has to be copied over to host (from guest) for clients to connect
- Port at which
dockerinside CDK running need to be mapped at host machine port for client to easily connect
Step -1 above is taken care by incorporating steps to generate certs and updating config in Vagrantfile
Step -3 is again taken care by specifying port mapping in Vagrantfile with auto_correct option incase of port collision at host - There are few complexities (to be covered later) for libvirt provider however for Virutalbox it works as expected.
For solving problem stated in Step-2 there are following two options
- Use ssh and scp to copy over the certs from inside the box to host
- Need for Windows platform:
sshandscpavailability on host - vagrant-triggers plugin for triggering the
scpoperation as box is "provisioned"- Knowledge about the mapped
sshport at host machine for thescpcommand (working on making this automated)
- Knowledge about the mapped
- Setup VirtualBox guest additions inside guest and use vagrant synced folders feature. this will need
- a line in Vagrantfile for specifying the folder to be synced
- for atomic guest - vbox guest additions configured in a (SPC) container
- for non atomic guest - vbox guest additions configured
Note: plugin [vagrant-vbguest] (https://github.com/dotless-de/vagrant-vbguest) is another option which can set vbox guest additions for community non-atomic platforms (CentOS and Fedora)
##Hands-on NEW VERSION OF THIS DOC https://github.com/navidshaikh/testing-adb ###1. CDK + Eclipse with vbox guest additions for VirtualBox provider
vagrant plugin install vagrant-vbguest- Use following Vagrantfile
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure(2) do |config|
config.vm.box = "atomicapp/dev"
config.vm.network "forwarded_port", guest: 2376, host: 2379, auto_correct: true
if !Vagrant.has_plugin?("vagrant-vbguest")
puts "'vagrant-vbguest' plugin is required"
puts "This can be installed by running:"
puts
puts " vagrant plugin install vagrant-vbguest"
puts
exit
end
config.vm.provision "shell", inline: <<-SHELL
# =========================================================
# Generate Certs for running TLS enabled docker daemon
# Generate Certificates to use with the docker daemon
# Instructions sourced from http://docs.docker.com/articles/https/
# Get the certificate location, i.e. setting the DOCKER_CERT_PATH variable
. /etc/sysconfig/docker
# randomString from http://utdream.org/post.cfm/bash-generate-a-random-string
# modified to echo value
function randomString {
# if a param was passed, it's the length of the string we want
if [[ -n $1 ]] && [[ "$1" -lt 20 ]]; then
local myStrLength=$1;
else
# otherwise set to default
local myStrLength=8;
fi
local mySeedNumber=$$`date +%N`; # seed will be the pid + nanoseconds
local myRandomString=$( echo $mySeedNumber | md5sum | md5sum );
# create our actual random string
#myRandomResult="${myRandomString:2:myStrLength}"
echo "${myRandomString:2:myStrLength}"
}
# Get a temporary workspace
dir=`mktemp -d`
cd $dir
# Get a random password for the CA and save it
passfile=tmp.pass
password=$(randomString 10)
echo $password > $passfile
# Generate the CA
openssl genrsa -aes256 -passout file:$passfile -out ca-key.pem 2048
openssl req -new -x509 -passin file:$passfile -days 365 -key ca-key.pem -sha256 -out ca.pem -subj "/C=/ST=/L=/O=/OU=/CN=example.com"
# Generate Server Key and Sign it
openssl genrsa -out server-key.pem 2048
openssl req -subj "/CN=example.com" -new -key server-key.pem -out server.csr
# Allow from 127.0.0.1
extipfile=extfile.cnf
echo subjectAltName = IP:127.0.0.1 > $extipfile
openssl x509 -req -days 365 -in server.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out server-cert.pem -passin file:$passfile -extfile $extipfile
# Generate the Client Key and Sign it
openssl genrsa -out key.pem 2048
openssl req -subj '/CN=client' -new -key key.pem -out client.csr
extfile=tmp.ext
echo extendedKeyUsage = clientAuth > $extfile
openssl x509 -req -days 365 -in client.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out cert.pem -extfile $extfile -passin file:$passfile
# Clean up
# set the cert path as configured in /etc/sysconfig/docker
## Move files into place
mv ca.pem $DOCKER_CERT_PATH
mv server-cert.pem $DOCKER_CERT_PATH
mv server-key.pem $DOCKER_CERT_PATH
# since the default user is vagrant and it can run docker without sudo
CLIENT_SIDE_CERT_PATH=/home/vagrant/.docker
mkdir -p $CLIENT_SIDE_CERT_PATH
cp $DOCKER_CERT_PATH/ca.pem $CLIENT_SIDE_CERT_PATH
mv cert.pem key.pem $CLIENT_SIDE_CERT_PATH
chown vagrant:vagrant $CLIENT_SIDE_CERT_PATH
chmod 0444 $CLIENT_SIDE_CERT_PATH/ca.pem
chmod 0444 $CLIENT_SIDE_CERT_PATH/cert.pem
chmod 0444 $CLIENT_SIDE_CERT_PATH/key.pem
chown vagrant:vagrant $CLIENT_SIDE_CERT_PATH/ca.pem
chown vagrant:vagrant $CLIENT_SIDE_CERT_PATH/cert.pem
chown vagrant:vagrant $CLIENT_SIDE_CERT_PATH/key.pem
chmod -v 0400 $DOCKER_CERT_PATH/ca.pem $DOCKER_CERT_PATH/server-cert.pem $DOCKER_CERT_PATH/server-key.pem
## Remove remaining files
cd
echo rm -rf $dir
# ============= end of script for generating the certs for TLS enabled docker daemon===
sed -i.back '/OPTIONS=*/c\OPTIONS="--selinux-enabled -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --tlscacert=/etc/docker/ca.pem --tlscert=/etc/docker/server-cert.pem --tlskey=/etc/docker/server-key.pem --tlsverify"' /etc/sysconfig/docker
sudo systemctl restart docker
SHELL
# setup the synced folders
config.vm.synced_folder ".", "/home/vagrant/.docker/"
config.vbguest.no_remote = true
end-
Run
vagrant upthis will install the vbox guest additions inside the guest, (be patient) -
you should have the certs in your current directory
-
clients can make connection using
127.0.0.1:2379and using the certs present in local directory -
for testing with docker do (on host)
cp -r *.pem ~/.docker docker -H 127.0.0.1:2379 --tlsverify images
###2. CDK + Eclipse with scp and ssh for VirtualBox provider NEW VERSION OF THIS DOC https://github.com/navidshaikh/testing-adb
vagrant plugin install vagrant-triggers- Export following into a
Vagrantfile
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure(2) do |config|
config.vm.box = "atomicapp/dev"
config.vm.network "forwarded_port", guest: 2376, host: 2379, auto_correct: true
if !Vagrant.has_plugin?("vagrant-triggers")
puts "'vagrant-triggers' plugin is required"
puts "This can be installed by running:"
puts
puts " vagrant plugin install vagrant-triggers"
puts
exit
end
config.vm.provision "shell", inline: <<-SHELL
# =========================================================
# Generate Certs for running TLS enabled docker daemon
#!/bin/bash
# Todo: move the files into place
# Generate Certificates to use with the docker daemon
# Instructions sourced from http://docs.docker.com/articles/https/
# Get the certificate location, i.e. setting the DOCKER_CERT_PATH variable
. /etc/sysconfig/docker
# randomString from http://utdream.org/post.cfm/bash-generate-a-random-string
# modified to echo value
function randomString {
# if a param was passed, it's the length of the string we want
if [[ -n $1 ]] && [[ "$1" -lt 20 ]]; then
local myStrLength=$1;
else
# otherwise set to default
local myStrLength=8;
fi
local mySeedNumber=$$`date +%N`; # seed will be the pid + nanoseconds
local myRandomString=$( echo $mySeedNumber | md5sum | md5sum );
# create our actual random string
#myRandomResult="${myRandomString:2:myStrLength}"
echo "${myRandomString:2:myStrLength}"
}
# Get a temporary workspace
dir=`mktemp -d`
cd $dir
# Get a random password for the CA and save it
passfile=tmp.pass
password=$(randomString 10)
echo $password > $passfile
# Generate the CA
openssl genrsa -aes256 -passout file:$passfile -out ca-key.pem 2048
openssl req -new -x509 -passin file:$passfile -days 365 -key ca-key.pem -sha256 -out ca.pem -subj "/C=/ST=/L=/O=/OU=/CN=example.com"
# Generate Server Key and Sign it
openssl genrsa -out server-key.pem 2048
openssl req -subj "/CN=example.com" -new -key server-key.pem -out server.csr
# Allow from 127.0.0.1
extipfile=extfile.cnf
echo subjectAltName = IP:127.0.0.1 > $extipfile
openssl x509 -req -days 365 -in server.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out server-cert.pem -passin file:$passfile -extfile $extipfile
# Generate the Client Key and Sign it
openssl genrsa -out key.pem 2048
openssl req -subj '/CN=client' -new -key key.pem -out client.csr
extfile=tmp.ext
echo extendedKeyUsage = clientAuth > $extfile
openssl x509 -req -days 365 -in client.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out cert.pem -extfile $extfile -passin file:$passfile
# Clean up
# set the cert path as configured in /etc/sysconfig/docker
## Move files into place
mv ca.pem $DOCKER_CERT_PATH
mv server-cert.pem $DOCKER_CERT_PATH
mv server-key.pem $DOCKER_CERT_PATH
# since the default user is vagrant and it can run docker without sudo
CLIENT_SIDE_CERT_PATH=/home/vagrant/.docker
mkdir -p $CLIENT_SIDE_CERT_PATH
cp $DOCKER_CERT_PATH/ca.pem $CLIENT_SIDE_CERT_PATH
mv cert.pem key.pem $CLIENT_SIDE_CERT_PATH
chown vagrant:vagrant $CLIENT_SIDE_CERT_PATH
chmod 0444 $CLIENT_SIDE_CERT_PATH/ca.pem
chmod 0444 $CLIENT_SIDE_CERT_PATH/cert.pem
chmod 0444 $CLIENT_SIDE_CERT_PATH/key.pem
chown vagrant:vagrant $CLIENT_SIDE_CERT_PATH/ca.pem
chown vagrant:vagrant $CLIENT_SIDE_CERT_PATH/cert.pem
chown vagrant:vagrant $CLIENT_SIDE_CERT_PATH/key.pem
chmod -v 0400 $DOCKER_CERT_PATH/ca.pem $DOCKER_CERT_PATH/server-cert.pem $DOCKER_CERT_PATH/server-key.pem
## Remove remaining files
cd
echo rm -rf $dir
# ============= end of script for generating the certs for TLS enabled docker daemon===
sed -i.back '/OPTIONS=*/c\OPTIONS="--selinux-enabled -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --tlscacert=/etc/docker/ca.pem --tlscert=/etc/docker/server-cert.pem --tlskey=/etc/docker/server-key.pem --tlsverify"' /etc/sysconfig/docker
sudo systemctl restart docker
SHELL
# triggers
config.trigger.after :provision do
hport = `vagrant ssh-config`.split("\n ").find{|e| e.start_with?("Port") }.split(" ")[1]
run "scp -r -P #{hport} -o StrictHostKeyChecking=no -i .vagrant/machines/default/virtualbox/private_key [email protected]:/home/vagrant/.docker/* ."
end
end-
Run
vagrant up --provider virtualbox(in the same directory where Vagrantfile exist) -
Run
vagrant provision -
You should have the certs in your current directory which is created (or updated) as part of above step
-
To test the client connection to
dockerdaemon inside CDK: In the Vagrantfile at line no: 7, we have mapped host port2379todockerdaemon port (2376) inside CDK, which means that you can access the daemon at (host machine) 127.0.0.1:2379. Following is an example of connecting viadockerCLI to daemon over TLS enabled TCP connection# copy the certs (generated above) to proper place where `docker` (by default) look up cp -r *.pem ~/.docker docker -H 127.0.0.1:2379 --tlsverify images
Note: On host machine, if you have port 2379 in use, vagrant will try to auto_correct it and assign a different port. You need to keep an eye on the corrected port while vagrant is brigning the machine up. Note that this vagrant
auto_correctfeature works as expected for Virtualbox provider but for libvirt provider it does not auto_correct and does not even fail! -
To test out the connection with Eclipse kindly check https://www.eclipse.org/community/eclipse_newsletter/2015/june/article3.php and this Video by Xavier Coulon https://www.youtube.com/watch?v=RUgEgtLux8Q. More Eclipse Docker Tooling documentations are at <https://wiki.eclipse.org/Linux_Tools_Project/Docker_Tooling/User_Guide>
@maxandersen:
dockerdaemon inside vagrant box will not allow the clients to be connected without certs. Requesting to please file any further issues on the new version of this doc at repository https://github.com/navidshaikh/testing-adb