Skip to content

Instantly share code, notes, and snippets.

@wnz99

wnz99/aws-gcp-openvpn.MD

Last active January 27, 2020 08:38
Show Gist options
  • Save wnz99/982fdbb3c6a70f41958c1e9006243758 to your computer and use it in GitHub Desktop.
Save wnz99/982fdbb3c6a70f41958c1e9006243758 to your computer and use it in GitHub Desktop.
Download ZIP
aws-gcp-openvpn
Raw
aws-gcp-openvpn.MD

Connecting AWS and GCP with OpenVPN

This tutorial will explain how to set up OpenVPN to connect AWS and GCP to allow access from AWS VM to GCP VM.

This is a basic guide for learning purposes and assumes default network configurations on both clouds.

Environment

Our environment will have the following setup in the two clouds. For sake of simplicity each VM will have a public address, but such setup is not recommended in production.

AWS

  • vm-a: a EC2 instance running some service, private IP:

    172.31.23.254

  • vm-b: a EC2 instance running the OpenVPN client, private IP:

    172.31.20.98

GCP

  • vm-c: a CE instance running some service, private IP:

    10.128.0.2

  • vm-d: a CE instance running the OpenVPN server, private IP:

    10.128.0.3

At the end of this tutorial vm-a will be able to communicate with vm-d through the OpenVPN link.

AWS setup

Connect to your AWS console and follow these steps:

  • Create two Ubuntu 18.04 LTS EC2 instances and name them vm-a and vm-b. Assign them the default network and subnet.

  • Allow all incoming TCP, UDP and ICMP traffic to the VMs. This is not recommended in productions and we are only doing this to make thing easier for the purpose of this tutorial.

  • Disable Source/Destination check for instance vm-b. This step is very important because it will allow this VM to forward traffic coming from vm-a. More about this at:

    https://docs.aws.amazon.com/vpc/latest/userguide/VPC_NAT_Instance.html#EIP_Disable_SrcDestCheck

  • Take not of the VMs network. In our setup the two VMs are part of the following network:

172.31.0.0/16

GCP setup

Connect to your GCP console and follow there steps:

  • Create two compute instances and them vm-c and vm-b whit IP Forwarding enabled on the advance network configuration. This can only be done at this stage and cannot be changed once an instance is create.

  • Take not of the VMs subnet. In our setup the two VMs are part of the following subnet:

10.128.0.0/20

Additional AWS setup

Now that we know GCP subnet we will need to set appropriate routes in AWS network to direct traffic to GCP range of IPs through the VPN link.

So browse to AWS VPCs configuration page. Look for your VMs VPC in the list and select the route table under Main Route table column. In the Routes tab add the following route:

10.128.0.0/20 to vm-b instance.

What does this mean? We are telling AWS that any traffic directed to the subnet 10.128.0.0/20 should be sent to vm-c.

Let's check this is happening:

  • ssh into vm-a and vm-b
  • in vm-a run ping 10.128.0.2
  • in vm-b run tcpdump -n -i eth0 host 172.31.20.98 and 10.128.0.2 icmp

You should see something like the following:

listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
11:44:38.273083 IP 172.31.20.98 > 10.128.0.2: ICMP echo request, id 14716, seq 1953, length 64
11:44:39.297030 IP 172.31.20.98 > 10.128.0.2: ICMP echo request, id 14716, seq 1954, length 64

What is it happening?

When you do a ping to 10.128.0.2 on vm-a, the OS will look at the local routing table to figure out the next hop to reach 10.128.0.2. If no specific routes are set, the packet will be sent to the default gateway, which i, our case, is 172.31.16.1. You can check this by running ip route on the machine.

The packet will then reach the default gateway, which will again look at his own routing table to figure out where to send it. If it does not find a relevant route, the packet is dropped, however we have set the correct route so the packet will be sent to vm-b. First baby step done.

OpenVPN setup

We assume that on vm-d you have a standard OpenVPN server installation with network routing and individual credential keys for each client.

During this tutorial we will require to makes changes to you OpenVPN configuration and will try to explain why that it's necessary.

In this tutorial the VPN subnet will be:

server 10.8.0.0 255.255.255.0

It is strongly advised to add the following line to bot client and server config:

verb 6

This will be useful to debug and understand where the traffic is going.

Make sure that ip forwarding is enable on this machine and that the traffic to the network behind is :

sudo echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf
sudo sysctl -f

We change the packet source ip so that it looks like it is coming from our OpenVPN server

iptables -t nat -I POSTROUTING -o ens4 \
      -s 10.8.0.0/24 -j MASQUERADE

If everything is congifured correctly you should be able to ping vm-c from vm-b:

# From vm-b
ping 10.128.0.2

Configuration on vm-b AWS OpenVPN client

Now that our packets are routed to this machine, we need to explain the local system what to to with them. This happens in two steps:

  1. The system must be able to route packets. This requires enabling the packet forwarding feature in the Kernel:

    sudo echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf
sudo sysctl -f

  2. We need to tell the system that packets coming from 10.128.0.0/20 have to be routed through the OpenVPN link. This can be accomplished either by manually setting the appropriate route in the system or by configuring the OpenVPN server to push the route to the client. We will follow the second solution, so open your OpenVPN server configuration file and add the following line to advertise the 10.128.0.0/24 subnet to VPN clients as being accessible through the VPN

    push "route 10.128.0.0 255.255.240.0"

Restart both OpenVPN server and client and run ping 10.128.0.2 on vm-a. On vm-d, your OpenVPN server, check the server logs. You will find something like this:

Sun Jan 26 09:57:49 2020 us=330233 aws/3.11.40.64:38232 UDPv4 READ [108] from [AF_INET]3.11.40.64:38232: P_DATA_V2 kid=0 DATA len=107

Sun Jan 26 09:57:49 2020 us=330407 aws/3.11.40.64:38232 MULTI: bad source address from client [172.31.20.98], packet dropped

Sun Jan 26 09:57:50 2020 us=354247 aws/3.11.40.64:38232 UDPv4 READ [108] from [AF_INET]3.11.40.64:38232: P_DATA_V2 kid=0 DATA len=107

Sun Jan 26 09:57:50 2020 us=354419 aws/3.11.40.64:38232 MULTI: bad source address from client [172.31.20.98], packet dropped

So, the packets are reaching GCP through the OpenVPN link but they are being dropped. Let's see what's going on.

Configuration on vm-d GCP OpenVPN server

Serverfault.com at the rescue:

https://serverfault.com/a/695555

Which can be summarized with:

--iroute network [netmask]
Generate an internal route to a specific client. The netmask 
parameter, if omitted, defaults to 255.255.255.255.
This directive can be used to route a fixed subnet from the server 
to a particular client, regardless of where the client is 
connecting from. Remember that you must also add the route to the 
system routing table as well (such as by using the --route 
directive). The reason why two routes are needed is that the 
--route directive routes the packet from the kernel to OpenVPN. 
Once in OpenVPN, the --iroute directive routes to the specific 
client.

So, follow the steps below:

  • edit your OpenVPN server config file and add the following lines:

    client-config-dir ccd
route 172.31.0.0 255.255.0.0

    This instructs the kernel to hand packets to 172.31.0.0 255.255.0.0 to the OpenVPN server

  • create specific configuration files for the aws client (we assume that the common name is aws):

    cd /etc/openvpn
sudo mkdir ccd
sudo touch ccd/aws
echo "iroute 172.31.0.0 255.255.0.0" > ccd/aws

This instructs the OpenVPN server to route send the traffic for 172.31.0.0 255.255.0.0 to the aws client.

Restart the server and client and run the following command in vm-d while pinging 10.128.0.2 from AWS instance vm-a:

tcpdump -n -i ens4 host 172.31.20.98 and 10.128.0.2 icmp

You should get the following output:

listening on ens4, link-type EN10MB (Ethernet), capture size 262144 bytes
11:38:04.112837 IP 172.31.20.98 > 10.128.0.2: ICMP echo request, id 14716, seq 1568, length 64
11:38:05.136774 IP 172.31.20.98 > 10.128.0.2: ICMP echo request, id 14716, seq 1569, length 64

So, the traffic is reaching our OpenVPN server and is correctly routing it to the internal instance vm-c

However, are we receiving a reply to our ping? It looks like we are not.

Configuration on vm-b AWS OpenVPN client

Login into vm-c and run tcpdump -n -i ens4 host 172.31.20.98 and 10.128.0.2 icmp.

You should not see any output.

The reason is tha the internal GCP network policies do not allow instances to forward traffic from other sources. There are two solutions:

  1. To change the instance policy to allow traffic forwarding.

  2. Masquerading the traffic from that instance.

We will implement the second solution.

Login into vm-d, your OpenVPN server and add the following iptable rule:

iptables -t nat -I POSTROUTING -o ens4 \
      -s 172.31.0.0/16 -j MASQUERADE

Restart the ping from vm-a and log into vm-c, then run:

tcpdump -n -i ens4 host 10.128.0.3 and 10.128.0.2

You should see the following output:

listening on ens4, link-type EN10MB (Ethernet), capture size 262144 bytes
11:55:12.946894 IP 10.128.0.3 > 10.128.0.2: ICMP echo request, id 14754, seq 93, length 64
11:55:12.946935 IP 10.128.0.2 > 10.128.0.3: ICMP echo reply, id 14754, seq 93, length 64

And also getting a reply to your ping from vm-a.

Final note

If you had enabled IP Forwarding on GCP instances, you can also reach the AWS network from GCP by adding an internal network route (similarly to what we did on ASW) to direct traffic to the OpenVPN server.

Once done so, you should be able to ping vm-a from vm-c with:

ping 172.31.20.98
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment