cross node docker container communication

This Gist demos how to make the inter communication between docker containers ( cross node)

###inter communication through Birdge

suppose there are two nodes n1: 192.168.1.1 and n2: 192.168.1.2, in the node n1 start container c1 and in the node n2 start container c2.


  --------------------         ------------------------
 |                    |        |                        |
 |      --------      |        |      ---------         |
 |    |  c1     |     |        |     |  c2     |        |
 |     ---------      |        |      ---------         |
 | n1: 192.168.1.1    |        | n2: 192.168.1.2        |
  -------------------           ------------------------

####1, create brige interface br0 in n1 and n2

in node n1 create bridge br0

$ sudo ip link add br0 type bridge
$ sudo ip addr add 10.1.1.1/24 broadcast 10.1.255.255 dev br0
$ sudo ip link set dev br0 up

in the node n2 create bridge br0

$ sudo ip link add br0 type bridge
$ sudo ip addr add 10.1.2.1/24 broadcast 10.1.255.255 dev br0
$ sudo ip link set dev br0 up

####2, add route

in the node n1

# go to 10.1.2.0/24 network segment ( in node n2 ) through gateway 192.168.1.2( n2 address)
$ sudo route add -net 10.1.2.0/24 gw 192.168.1.2
$ sudo iptables -t nat -F POSTROUTING
$ sudo iptables -t nat -A POSTROUTING -s 10.1.1.0/24 ! -d 10.1.0.0/16 -j MASQUERADE

in the node n2

# go to 10.1.1.0/24 network segment ( in node n1 ) through gateway 192.168.1.1( n1 address)
$ sudo route add -net 10.1.1.0/24 gw 192.168.1.1
$ sudo iptables -t nat -F POSTROUTING
$ sudo iptables -t nat -A POSTROUTING -s 10.1.2.0/24 ! -d 10.1.0.0/16 -j MASQUERADE

####4, start docker daemon on bridge br0

if in ubuntu, add "-b=br0" to the DOCKER_OPTS in /etc/default/docker file

then restart the docker daemon in n1 and n2:

$ sudo service docker restart
or
$sudo systemctl restart docker

####3, start container and check ip address then in the n1, start a container c1

$ sudo docker -it busybox /bin/sh
/ # ifconfig
eth0      Link encap:Ethernet  HWaddr 02:42:0A:01:02:06
          inet addr:10.1.1.3  Bcast:0.0.0.0  Mask:255.255.255.0
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:3 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:258 (258.0 B)  TX bytes:418 (418.0 B)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

then in the n2, start a container c2

$ sudo docker -it busybox /bin/sh
/ # ifconfig
eth0      Link encap:Ethernet  HWaddr 02:42:0A:01:02:06
          inet addr:10.1.2.3  Bcast:0.0.0.0  Mask:255.255.255.0
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:3 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:258 (258.0 B)  TX bytes:418 (418.0 B)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

####4, ping each other

in container c1, ping container c2:

#ping 10.1.2.3
PING 10.1.2.3 (10.1.2.3): 56 data bytes
64 bytes from 10.1.2.3: seq=0 ttl=62 time=1.116 ms

in container c2, oing container c1:

#ping 10.1.1.3
PING 10.1.1.3 (10.1.1.3): 56 data bytes
64 bytes from 10.1.1.3: seq=0 ttl=62 time=1.116 ms

ochinchina/docker_container_inter_communication.md