Skip to content

Instantly share code, notes, and snippets.

@d34dh0r53
Created August 26, 2015 18:43
Show Gist options
  • Save d34dh0r53/ecb1dc20dca65367b8a8 to your computer and use it in GitHub Desktop.
Save d34dh0r53/ecb1dc20dca65367b8a8 to your computer and use it in GitHub Desktop.
Swift Global Clustering Notes

As a prerequisite the deployment will need to have their VPN tunnels configured with all relevant VLANs, tunnels and routes ready to place on the host machines. I think that the host machine will also need to have ip_forward enabled:

echo 1 > /proc/sys/net/ipv4/ip_forward

br-repl

On any host that will be part of the mesh will need a br-repl bridge created with the host-local IPs on it. Any tunnels/VLANs/etc that need to be on the host belong in this bridge. An example I wrote for the host networking chapter

  • Replication provider br-repl:
    • Optional
    • Provides access to inter-region networking mesh.
    • Manually created and is given the host-local gateway IP(s). Also attaches to eth13 in each associated container.
    • Static routes to the inter-region network from the containers should be defined in openstack_user_variables.yml.

Here is an example br-repl for a simple point to point VPN:

auto br-repl
iface br-repl inet static
    bridge_stp off
    bridge_waitport 0
    bridge_fd 0
    bridge_ports none
    address <host-local gateway>
    netmask <netmask>
    # Create veth pair, ignore if already exists
    pre-up ip link add br-repl-veth type veth peer name eth13 || true
    # Set both ends UP
    pre-up ip link set eth13 up
    pre-up ip link set br-repl-veth up
    # Add the routes
    post-up ip route add <mesh CIDr> via <host-local gateway> onlink
    # Delete veth pair on DOWN
    post-down ip link del br-repl-veth || true
    bridge_ports br-repl-veth

I think this is where the deployer would add aditional interfaces into the the bridge via bridge_ports.

Once the host side networking has been completed the OSAD configuration can take place. Basically all that needs to happen is to add a static_routes section to etc/openstack_deploy/openstack_user_config.yml:

- network:
  container_bridge: "br-repl"
  container_type: "veth"
  container_interface: "eth13"
  ip_from_q: "repl"
  type: "raw"
  group_binds:
    - swift_proxy
  static_routes:
    - cidr: <mesh cidr>
      gateway: <ip on br-repl>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment