Skip to content

Instantly share code, notes, and snippets.

@arcolife
Last active September 8, 2018 11:09
Show Gist options
  • Save arcolife/6db55d44695195c20cb5ade48f919189 to your computer and use it in GitHub Desktop.
Save arcolife/6db55d44695195c20cb5ade48f919189 to your computer and use it in GitHub Desktop.
iptables hack for openstack (no floating ip for keystone)

CFME side

iptables -A OUTPUT -t nat -d 192.0.2.1 -j DNAT --to-destination 10.16.154.6
iptables -A OUTPUT -t nat -d 172,21,0.10 -j DNAT --to-destination 10.16.154.9

OSP side

Overcloud (heat admin):

  • Floating IP: 10.16.154.141
  • Bridge: br-ex
  • Private IP (Keystone): 192.0.2.11

Undercloud:

  • Floating IP: 10.16.154.126
  • Bridge: em1
  • Private IP: 192.0.2.1

^check which interface has a floating IP for both undercloud and overcloud

Repeat following steps:

  • after sourcing stackrc for stack user of undercloud,
  • and then sourcing overcloudrc for head-admin user of overcloud-controller:

list all endpoint ports

$ for i in `openstack endpoint list | awk -F'| ' '{ print $2}' | sed 1,3d | sed '/^$/d'`; do openstack endpoint show $i | grep admin; done > endpoint_ports

extract endpoint ports

$ cat endpoint_ports |  sed '/^$/d' | awk -F':' '{ print $3}' |  sed '/^$/d' | awk -F'/| ' '{ print $1}' > ports_new

apply settings

# undercloud
$ for i in `cat ports_new`; do sudo iptables -t nat -A PREROUTING -i em1 -p tcp --dport $i -j DNAT --to-destination 192.0.2.1:$i; done
# overcloud
$ for i in `cat ports_new`; do sudo iptables -t nat -A PREROUTING -i br-ex -p tcp --dport $i -j DNAT --to-destination 192.0.2.11:$i; done

do an additional step for 5000 port (not included in ports_new from above steps)

# undercloud
$ iptables -t nat -A PREROUTING -i em1 -p tcp --dport 5000 -j DNAT --to-destination 192.0.2.1:5000
# overcloud
$ iptables -t nat -A PREROUTING -i br-ex -p tcp --dport 5000 -j DNAT --to-destination 192.0.2.11:5000

optionally, if you messup, delete ports from iptables like this:

# for undercloud
for i in {1..13}; do sudo iptables -t nat -D PREROUTING 4; done

# overcloud
for i in {1..13}; do sudo iptables -t nat -D PREROUTING 2; done
@arcolife
Copy link
Author

undercloud iptables

[stack@gprfc043 ~]$ sudo iptables -t nat -L -n -v --line-numbers
Chain PREROUTING (policy ACCEPT 30447 packets, 2510K bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1     153K 9191K REDIRECT   tcp  --  br-ctlplane *       0.0.0.0/0            169.254.169.254      tcp dpt:80 redir ports 8775
2     194K   20M nova-api-PREROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
3     156K 9903K DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL
4        0     0 DNAT       tcp  --  em1    *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8774 to:192.0.2.1:8774
5        0     0 DNAT       tcp  --  em1    *       0.0.0.0/0            0.0.0.0/0            tcp dpt:6385 to:192.0.2.1:6385
6        0     0 DNAT       tcp  --  em1    *       0.0.0.0/0            0.0.0.0/0            tcp dpt:9000 to:192.0.2.1:9000
7        0     0 DNAT       tcp  --  em1    *       0.0.0.0/0            0.0.0.0/0            tcp dpt:9696 to:192.0.2.1:9696
8        0     0 DNAT       tcp  --  em1    *       0.0.0.0/0            0.0.0.0/0            tcp dpt:35357 to:192.0.2.1:35357
9        0     0 DNAT       tcp  --  em1    *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8888 to:192.0.2.1:8888
10       0     0 DNAT       tcp  --  em1    *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8080 to:192.0.2.1:8080
11       0     0 DNAT       tcp  --  em1    *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8777 to:192.0.2.1:8777
12       0     0 DNAT       tcp  --  em1    *       0.0.0.0/0            0.0.0.0/0            tcp dpt:5050 to:192.0.2.1:5050
13       0     0 DNAT       tcp  --  em1    *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8989 to:192.0.2.1:8989
14       0     0 DNAT       tcp  --  em1    *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8004 to:192.0.2.1:8004
15       0     0 DNAT       tcp  --  em1    *       0.0.0.0/0            0.0.0.0/0            tcp dpt:9292 to:192.0.2.1:9292
16       0     0 DNAT       tcp  --  em1    *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8042 to:192.0.2.1:8042
17       0     0 DNAT       tcp  --  em1    *       0.0.0.0/0            0.0.0.0/0            tcp dpt:5000 to:192.0.2.1:5000

Chain INPUT (policy ACCEPT 58213 packets, 4176K bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 124K packets, 8152K bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1     712K   47M nova-api-OUTPUT  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
2     452K   27M DOCKER     all  --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 124K packets, 8152K bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1     711K   47M BOOTSTACK_MASQ  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
2     712K   47M nova-api-POSTROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
3     712K   47M nova-postrouting-bottom  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
4        0     0 MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0           
5        0     0 MASQUERADE  all  --  *      eth0    192.0.2.0/24         0.0.0.0/0           

Chain BOOTSTACK_MASQ (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 RETURN     all  --  *      *       192.0.2.0/24         192.168.122.1       
2        2   684 MASQUERADE  all  --  *      *       192.0.2.0/24        !192.0.2.0/24        

Chain DOCKER (2 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 RETURN     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0           

Chain nova-api-OUTPUT (1 references)
num   pkts bytes target     prot opt in     out     source               destination         

Chain nova-api-POSTROUTING (1 references)
num   pkts bytes target     prot opt in     out     source               destination         

Chain nova-api-PREROUTING (1 references)
num   pkts bytes target     prot opt in     out     source               destination         

Chain nova-api-float-snat (1 references)
num   pkts bytes target     prot opt in     out     source               destination         

Chain nova-api-snat (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1     712K   47M nova-api-float-snat  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain nova-postrouting-bottom (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1     712K   47M nova-api-snat  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment