Skip to content

Instantly share code, notes, and snippets.

@wallnerryan

wallnerryan/ctddemoswarmmachine.md

Last active August 29, 2015 14:27
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save wallnerryan/16f99773271e32550daf to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save wallnerryan/16f99773271e32550daf to your computer and use it in GitHub Desktop.
Download ZIP
Walkthrough of machine swarm on virtualbox on Mac
Raw
ctddemoswarmmachine.md

Modified Version of these Instructions for Mac OSX and Virtual Box

https://github.com/docker/compose/blob/master/SWARM.md https://docs.docker.com/machine/#using-docker-machine-with-docker-swarm

Pre-requisites

Docker [https://github.com/docker/docker/releases]

(for mac)

curl -L https://get.docker.com/builds/Darwin/x86_64/docker-1.8.0  > /usr/local/bin/docker
chmod +x /usr/local/bin/docker

Docker-Machine [https://github.com/docker/machine/releases]

curl -L https://github.com/docker/machine/releases/download/v0.4.0/docker-machine_linux-amd64 > /usr/local/bin/docker-machine
chmod +x /usr/local/bin/docker-machine

Docker-Compose [https://github.com/docker/compose/releases]

curl -L https://github.com/docker/compose/releases/download/1.4.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose

VirtualBox [https://www.virtualbox.org/wiki/Downloads]

Docker Toolbox

If you dont want to install all of this manually, please install (Docker Toolbox)[https://www.docker.com/toolbox] which allows you to select and install

  • Docker
  • Docker Compose
  • Docker Machine
  • Kitematic
  • Virtualbox

Machine + Swarm + (Compose)

  • If VPN or on EMC Network Download ISO [https://github.com/boot2docker/boot2docker/releases], place in /Users/<Username>/.docker/machine/cache/ otherwise machine will hang due to firewall.

  • docker-machine create -d virtualbox local

    • (If it hangs here, most likely a Cisco VPN or Firewall issue, try adding a route to the subnet virtualbox is using)

    • (sudo route -n add -net 192.168.99.0/24 -interface vboxnet5)

      • (sudo route change -net 192.168.99.0/24 -interface vboxnet5)
      • If still not working see ( sudo ipfw -a list)

    • Look for for deny ip from any to any in

    • 00024 725 195481 deny ip from any to any

    • 01200 0 0 deny ip from any to any in

      • (sudo ipfw delete 01200)
      • (sudo ipfw delete 00024)

    • Routes should be fixed

    • netstat -nr | grep "^192.*"

    • 192.168.99 link#15 UCS 1 0 vboxnet

    • 192.168.99.100 8:0:27:92:43:41 UHLWIi 1 25 vboxnet 1121

    • 192.168.99.101 8:0:27:0:d3:ea UHLWIi 1 5 vboxnet 1197

  • eval "$(docker-machine env local)"

If your on the EMC network We need to add EMC Certs to each machine if we want to pull docker images inside EMC

  • sub1. Open http://gso.corp.emc.com/installupdatedcerts.aspx and Download EMCs SSL Decryption certificate.
  • sub2. Convert the certificate to a PEM file with openssl x509 -in ~/Downloads/EMC\ SSL.cer -out EMC_SSL.pem
  • sub.2.5 cat EMC_SSL.pem | sudo tee -a /etc/ssl/certs/ca-certificates.crt
  • sub3. docker-machine ssh local
  • sub4. ps ax | grep docker (find PID) and “sudo kill -9 <PID>”
  • sub5. (Restart daemon with exact command as above.
    • sudo /usr/local/bin/docker -d -D -g /var/lib/docker -H unix:// -H tcp://0.0.0.0:2376 --label provider=virtualbox --tlsverify --tlscacert=/var/lib/boot2docker/ca.pem --tlscert=/var/lib/boot2docker/server.pem --tlskey=/var/lib/boot2docker/server-key.pem -s aufs &
  • sub6. exit
Now we should be good to go.
  • docker run swarm create (take not of the key created, and example looks like this 96295db29e0ad68eab3ac5032132206d and will be used in the below commands, please replace with YOUR key when going through these examples.)
  • docker-machine ls (See IP Addresses) (Run the agent, with your ABOVE key)
  • docker run —detach swarm join --advertise=192.168.99.102:2376 token://96295db29e0ad68eab3ac5032132206d
  • docker-machine create -d virtualbox swarm-master
  • eval "$(docker-machine env swarm-master)"
    • (Repeat Above Step #2.5,3,4,5,6 for EMC Certs if needed)
  • docker-machine ls (See IP Addresses)
  • docker run —detach swarm join --advertise=192.168.99.104:2376 token://96295db29e0ad68eab3ac5032132206d

(Now we need to pass in boot2docker certs with a volume and run the swarm manager)

  • docker run --detach -p 2375:2375 -v /var/lib/boot2docker/:/certs/ swarm manage --tlsverify --tlscacert=/certs/ca.pem --tlscert=/certs/server.pem --tlskey=/certs/server-key.pem -H tcp://0.0.0.0:2375 token://96295db29e0ad68eab3ac5032132206d
Now we can run docker commands against our small 2 node swarm cluster.

(192.168.99.104 is the virtualbox/docker-machine ‘swarm-master’ node’s address)

→ docker -H tcp://192.168.99.104:2375 info
Containers: 13
Images: 2
Storage Driver: 
Role: primary
Strategy: spread
Filters: affinity, health, constraint, port, dependency
Nodes: 2
 local: 192.168.99.102:2376
  └ Containers: 3
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 1.022 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=4.0.7-boot2docker, operatingsystem=Boot2Docker 1.7.1 (TCL 6.3); master : c202798 - Wed Jul 15 00:16:02 UTC 2015, provider=virtualbox, storagedriver=aufs
 swarm-master: 192.168.99.104:2376
  └ Containers: 10
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 1.022 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=4.0.7-boot2docker, operatingsystem=Boot2Docker 1.7.1 (TCL 6.3); master : c202798 - Wed Jul 15 00:16:02 UTC 2015, provider=virtualbox, storagedriver=aufs
Execution Driver: 
Kernel Version: 
Operating System: 
CPUs: 2
Total Memory: 2.043 GiB
Name: f4ad1d74e965
ID:

Wharfee

A cool tool to use for working with the DockerCLI is Wharfee https://wharfee.com or https://github.com/j-bennet/wharfee

Install
sudo easy_install http://github.com/j-bennet/wharfee/tarball/master
Use

Wharfee auto-completes CLI for you so this is pretty nice to get to know the CLI.

  • To use against a single Docker API Deamon use the eval command below.
  • To use against the Swarm Master we created earlier export the host and tls verify environment variable to the swarm-master
→ export DOCKER_HOST=tcp://192.168.99.104:2375
→ export DOCKER_TLS_VERIFY=0

Example of wharfee agianst single endpoint swarm-master

$ eval "$(docker-machine env swarm-master)"
$ wharfee
wharfee>
wharfee> ps
Status       Created       Image    Id           Command                          Names                Ports
-----------  ------------  -------  -----------  -------------------------------  -------------------  ----------------------
Up 20 hours  22 hours ago  swarm    fb6c602a2ca  /swarm join --advertise=192.168  condescending_kilby  2375/tcp
Up 20 hours  22 hours ago  swarm    f4ad1d74e96  /swarm manage --tlsverify --tls  nostalgic_thompson   0.0.0.0:2375->2375/tcp
wharfee>

Example of wharfee agianst single enrpoint local

$ eval "$(docker-machine env local)"
$ wharfee
wharfee>
wharfee> ps
Status         Created       Image         Id           Command                          Names               Ports
-------------  ------------  ------------  -----------  -------------------------------  ------------------  ---------------------
Up 53 seconds  an hour ago   httpd:latest  1cb106e9fa3  httpd-foreground                 myhttpd             0.0.0.0:32770->80/tcp
Up 21 hours    22 hours ago  swarm         1090861f98e  /swarm join --advertise=192.168  stupefied_lovelace  2375/tcp
wharfee>

Example of wharfee agianst the exposed swarm cluster manager

→ export DOCKER_HOST=tcp://192.168.99.104:2375
→ export DOCKER_TLS_VERIFY=0
$ wharfee
wharfee>
wharfee> ps
Status             Created      Image         Id           Command           Names                              SizeRw    SizeRootFs  Ports
-----------------  -----------  ------------  -----------  ----------------  -------------------------------  --------  ------------  ----------------------------
Up 2 seconds       an hour ago  httpd:latest  5ccba58e94b  httpd-foreground  local/myhttpd6                          0             0  192.168.99.102:32769->80/tcp
Up About a minute  an hour ago  httpd:latest  b90f4e120e2  httpd-foreground  local/myhttpd5                          0             0  192.168.99.102:32768->80/tcp
Up 9 minutes       an hour ago  httpd:latest  0e9cfb10c44  httpd-foreground  swarm-master/myhttpd-exposed-sp         0             0  :0->80/tcp
Up 10 minutes      an hour ago  httpd:latest  7c3137fa6ac  httpd-foreground  swarm-master/myhttpd-exposed-sp         0             0  :0->80/tcp
Up 11 minutes      an hour ago  httpd:latest  7107716fdd1  httpd-foreground  swarm-master/myhttpd-exposed-sp         0             0  :0->80/tcp
Up 13 minutes      an hour ago  httpd:latest  f4caf55dfe0  httpd-foreground  swarm-master/myhttpd-exposed            0             0  192.168.99.104:32768->80/tcp
Up 14 minutes      an hour ago  httpd:latest  6369cb9e87c  httpd-foreground  swarm-master/myhttpd                    0             0  :0->80/tcp

Here are some examples of auto-completion, TAB works to complete as well.

wharfee> p 
           pause   
           port   
           ps     
           pull   
           push   

wharfee> ps 
             -a/--all     
             --before     
             -h/--help    
             -l/--latest  
             --no-trunc   
             -q/--quiet   
             --since      

wharfee> run -          
               --expose                   
               --link                     
               --name                     
               --rm                       
               --volumes-from             
               -P/--publish-all           
               -a/--attach
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment