UPDATE

Moved to https://github.com/oddmario/GRE-setup-guide
Or alternatively, a similar WireGuard guide is available at https://github.com/oddmario/WireGuard-setup-guide

Setup a GRE tunnel between two Linux servers

TL;DR network engineering is hard 🫠

Server A

It is the "GRE VPS" which we are going to use the IP address of instead of the IP address of Server B.

One recommended provider for Server A is BuyVM.net [especially with their DDoS protected IPs]

Server B

It is the "backend server" or the destination server. i.e. the server which we are trying to hide/protect the IP address of.

This guide covers

Setting up a GRE tunnel to link between two Linux servers (server A and server B)
Setting up the proper routing to make server A forward all the traffic to & from server B through the tunnel.

If you would like to use WireGuard (or even OpenVPN) instead of a GRE tunnel to link between the two servers, you can absolutely give that a go!

Generally, we just need a way to link between the two servers (either GRE, WireGuard or even OpenVPN). Then the rest of the commands to setup the routing through iproute2 and iptables should be similar.

Requirements

Server A needs to have a primary public IP address that we are going to use as the peer address for our GRE tunnels.

And you CAN NOT use the primary IP address of server A as a GRE tunnel since it will be the IP used by server B to connect to server A.

So in order to start GRE tunneling, you need additional IPs linked to server A by the provider of it.

`makeGRE.sh` on Server A

#!/bin/bash

# This script is placed on the GRE VPS

#
# Variables
#

GRE_VPS_MAIN_IP="[the main public ip address of the gre vps here]"
GRE_VPS_IP="[the public ip address of the gre vps that you are attempting to forward to the backend server]" # This cannot be the same value of GRE_VPS_MAIN_IP. check the requirements at the top of the guide for more information
BACKEND_IP="[backend server public ip address here]"
GRE_VPS_IP_MAIN_INTERFACE="eth0"

GRE_TUNNEL_INTERFACE_NAME="gre1"
GRE_TUNNEL_GREVPS_IP="192.168.168.1"
GRE_TUNNEL_BACKEND_IP="192.168.168.2"
GRE_TUNNEL_KEY="1"

# ----------------------------------

sysctl -p
systemctl stop firewalld
systemctl disable firewalld
modprobe tcp_bbr
tc qdisc replace dev $GRE_VPS_IP_MAIN_INTERFACE root fq
ip link set $GRE_VPS_IP_MAIN_INTERFACE txqueuelen 15000
modprobe ip_gre
iptables -F
iptables -P FORWARD ACCEPT
sysctl -w net.ipv4.conf.$GRE_VPS_IP_MAIN_INTERFACE.proxy_arp=1
# https://www.linuxquestions.org/questions/linux-networking-3/how-to-turn-on-keepalive-on-linux-interface-4175541981/#post6320771
sysctl -w net.ipv4.conf.default.accept_local=1
sysctl -w net.ipv4.conf.all.accept_local=1

ip tunnel add $GRE_TUNNEL_INTERFACE_NAME mode gre local $GRE_VPS_MAIN_IP remote $BACKEND_IP ttl 255 key $GRE_TUNNEL_KEY
ip link set $GRE_TUNNEL_INTERFACE_NAME up

ip addr add $GRE_TUNNEL_GREVPS_IP/30 dev $GRE_TUNNEL_INTERFACE_NAME
ip route add $GRE_VPS_IP/32 via $GRE_TUNNEL_BACKEND_IP

tc qdisc replace dev $GRE_TUNNEL_INTERFACE_NAME root fq
ip link set $GRE_TUNNEL_INTERFACE_NAME txqueuelen 15000
ethtool -K $GRE_TUNNEL_INTERFACE_NAME gro off gso off tso off
ethtool -K $GRE_VPS_IP_MAIN_INTERFACE gro off gso off tso off

`delGRE.sh` on Server A

#!/bin/bash

# This script is placed on the GRE VPS

#
# Variables
#

GRE_VPS_IP="[the public ip address of the gre vps that you are attempting to forward to the backend server]"

GRE_TUNNEL_INTERFACE_NAME="gre1"
GRE_TUNNEL_GREVPS_IP="192.168.168.1"
GRE_TUNNEL_BACKEND_IP="192.168.168.2"

# ----------------------------------

ip route del $GRE_VPS_IP/32 via $GRE_TUNNEL_BACKEND_IP
ip addr del $GRE_TUNNEL_GREVPS_IP/30 dev $GRE_TUNNEL_INTERFACE_NAME
ip link set $GRE_TUNNEL_INTERFACE_NAME down
ip tunnel del $GRE_TUNNEL_INTERFACE_NAME

`/etc/sysctl.conf` on Server A

net.ipv4.ip_forward=1
# Disabling IPv6 below is optional
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1

fs.file-max = 2097152
fs.inotify.max_user_instances=1048576
fs.inotify.max_user_watches=1048576
fs.nr_open=1048576
fs.aio-max-nr = 1048576

net.core.somaxconn=65535
net.core.netdev_max_backlog=16384
net.core.dev_weight = 64
net.ipv4.ip_local_port_range = 1024 65535

net.nf_conntrack_max=1000000
net.netfilter.nf_conntrack_max=1000000

net.ipv4.tcp_max_tw_buckets=1440000
net.unix.max_dgram_qlen = 50
net.ipv4.neigh.default.proxy_qlen = 96
net.ipv4.neigh.default.unres_qlen = 6

net.ipv4.tcp_congestion_control = bbr
net.core.default_qdisc = fq

net.ipv4.tcp_notsent_lowat = 16384

# https://serverfault.com/a/359232/554686
net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.default.rp_filter=0
net.ipv4.conf.all.accept_redirects=0
net.ipv4.conf.default.accept_redirects=0

net.ipv4.tcp_mtu_probing=1

net.ipv4.route.flush = 1
net.ipv6.route.flush = 1

`makeGRE.sh` on Server B

#!/bin/bash

# This script is placed on the backend server

#
# Variables
#

GRE_VPS_MAIN_IP="[the main public ip address of the gre vps here]"
GRE_VPS_IP="[the public ip address of the gre vps that you are attempting to forward to the backend server]" # This cannot be the same value of GRE_VPS_MAIN_IP. check the requirements at the top of the guide for more information
BACKEND_IP="[backend server public ip address here]"

GRE_TUNNEL_INTERFACE_NAME="gre1"
GRE_TUNNEL_GREVPS_IP="192.168.168.1"
GRE_TUNNEL_BACKEND_IP="192.168.168.2"
GRE_TUNNEL_RTTABLES_ID="100"
GRE_TUNNEL_RTTABLES_NAME="GRE"
GRE_TUNNEL_KEY="1"

# ----------------------------------

modprobe ip_gre

# https://www.linuxquestions.org/questions/linux-networking-3/how-to-turn-on-keepalive-on-linux-interface-4175541981/#post6320771
sysctl -w net.ipv4.conf.default.accept_local=1
sysctl -w net.ipv4.conf.all.accept_local=1

if ! grep -Fxq "$GRE_TUNNEL_RTTABLES_ID $GRE_TUNNEL_RTTABLES_NAME" /etc/iproute2/rt_tables
then
     echo "$GRE_TUNNEL_RTTABLES_ID $GRE_TUNNEL_RTTABLES_NAME" >> /etc/iproute2/rt_tables
fi

ip tunnel add $GRE_TUNNEL_INTERFACE_NAME mode gre local $BACKEND_IP remote $GRE_VPS_MAIN_IP ttl 255 key $GRE_TUNNEL_KEY
ip link set $GRE_TUNNEL_INTERFACE_NAME up

ip addr add $GRE_TUNNEL_BACKEND_IP/30 dev $GRE_TUNNEL_INTERFACE_NAME
ip addr add $GRE_VPS_IP/32 dev $GRE_TUNNEL_INTERFACE_NAME

ip rule add from $GRE_VPS_IP lookup $GRE_TUNNEL_RTTABLES_NAME
ip route add default via $GRE_TUNNEL_GREVPS_IP table $GRE_TUNNEL_RTTABLES_NAME

iptables -o $GRE_TUNNEL_INTERFACE_NAME -t nat -I POSTROUTING -j SNAT --to-source $GRE_VPS_IP

# dns servers are required otherwise all dns resolutions will fail
echo 'nameserver 1.1.1.1' > /etc/resolv.conf
echo 'nameserver 1.0.0.1' >> /etc/resolv.conf

tc qdisc replace dev $GRE_TUNNEL_INTERFACE_NAME root fq
ip link set $GRE_TUNNEL_INTERFACE_NAME txqueuelen 15000
ethtool -K $GRE_TUNNEL_INTERFACE_NAME gro off gso off tso off

`delGRE.sh` on Server B

#!/bin/bash

# This script is placed on the backend server

#
# Variables
#

GRE_VPS_MAIN_IP="[the main public ip address of the gre vps here]"
GRE_VPS_IP="[the public ip address of the gre vps that you are attempting to forward to the backend server]" # This cannot be the same value of GRE_VPS_MAIN_IP. check the requirements at the top of the guide for more information

GRE_TUNNEL_INTERFACE_NAME="gre1"
GRE_TUNNEL_GREVPS_IP="192.168.168.1"
GRE_TUNNEL_BACKEND_IP="192.168.168.2"
GRE_TUNNEL_RTTABLES_NAME="GRE"

# ----------------------------------

iptables -t nat -D POSTROUTING -o $GRE_TUNNEL_INTERFACE_NAME -j SNAT --to-source $GRE_VPS_IP

ip rule del from $GRE_VPS_IP lookup $GRE_TUNNEL_RTTABLES_NAME
ip route del default via $GRE_TUNNEL_GREVPS_IP table $GRE_TUNNEL_RTTABLES_NAME
ip addr del $GRE_VPS_IP/32 dev $GRE_TUNNEL_INTERFACE_NAME
ip addr del $GRE_TUNNEL_BACKEND_IP/30 dev $GRE_TUNNEL_INTERFACE_NAME
ip link set $GRE_TUNNEL_INTERFACE_NAME down
ip tunnel del $GRE_TUNNEL_INTERFACE_NAME

Notes

📌 each individual note is prefixed with a number. any dotted points are sub-points of a note.

On the GRE VPS:

It is recommended to use AlmaLinux
Make sure the system is up to date (dnf update)
Disable SELinux permanently

Add this to /etc/security/limits.conf:

* soft nproc 1048576
* hard nproc 1048576
* soft nofile 1048576
* hard nofile 1048576
* soft stack 1048576
* hard stack 1048576
* soft memlock unlimited
* hard memlock unlimited

Reboot the VPS after updating the system & disabling SELinux

A bad provider for the GRE tunnel will cause packet loss. An example of that is Aeza.net. See https://lowendtalk.com/discussion/192513/aeza-sweden-and-probably-other-locations-network-issues
Setting the incorrect MTU for the gre1 interface will cause packet loss. It is recommended to always keep the default MTU values set by the provider and Linux.
If you are facing issues after setting the GRE tunnel up, try disabling the firewall (ufw/firewalld) on the destination (backend) server [if it's enabled].

If this solves the problem but you would like to keep your firewall enabled, make sure the public IP address(es) of the GRE VPS and the private IP address(es) of the GRE VPS on the GRE tunnel (e.g. 192.168.168.1) are trusted on the firewall of the backend server.
⚠️ Make sure that NO additional IP addresses are linked to the operating system of the GRE VPS. Just keep the main IP address linked and that's it. Yes the GRE tunnels will magically make use of the additional IPs work even when they are not linked to the OS of the GRE VPS.

To make sure of this, you can use ip a to ensure that the main interface (e.g. eth0) has ONLY the main IP address of the GRE VPS linked to it.

In case there are multiple IPs linked (for example as eth0:1, eth0:2, etc [or even on network interfaces with different names such as eth1]), you need to remove them first before attempting to setup ANY GRE tunnels.

On AlmaLinux, the configuration files of any additional IPs are mostly found at /etc/sysconfig/network-scripts/

So for example if you have eth0:1 and eth0:2 configured, you can remove them using:
```
rm -rf /etc/sysconfig/network-scripts/ifcfg-eth0:1
rm -rf /etc/sysconfig/network-scripts/ifcfg-eth0:2
```
Then restart the networking service using systemctl restart NetworkManager.service && sleep 5 && nmcli networking off && nmcli networking on

⚠️ NOTE: You must restart your GRE tunnel (or all of your tunnels if you have multiple) after restarting the networking. This can be done by ./delGRE.sh && ./makeGRE.sh [make sure to do the same for all your GRE tunnels if you have multiple scripts].
If you have multiple IP addresses on the GRE VPS and you would like to use them to forward either to multiple different backend servers or to the same backend server, you can create multiple GRE tunnels.

On both the GRE VPS (Server A) and the backend server (Server B), create new makeGRE-2.sh and delGRE-2.sh files so we can create new GRE setup scripts. The content of the files should be the same scripts that are at the top of this guide.

Then edit this configurable part on the new scripts:
```
GRE_TUNNEL_INTERFACE_NAME="gre1"
GRE_TUNNEL_GREVPS_IP="192.168.168.1"
GRE_TUNNEL_BACKEND_IP="192.168.168.2"
GRE_TUNNEL_RTTABLES_ID="100"
GRE_TUNNEL_RTTABLES_NAME="GRE"
GRE_TUNNEL_KEY="1"
```
to be:
```
GRE_TUNNEL_INTERFACE_NAME="gre2"
GRE_TUNNEL_GREVPS_IP="192.168.169.1" # NOTE: uses 169 instead of 168
GRE_TUNNEL_BACKEND_IP="192.168.169.2" # NOTE: uses 169 instead of 168
GRE_TUNNEL_RTTABLES_ID="200"
GRE_TUNNEL_RTTABLES_NAME="GRE2"
GRE_TUNNEL_KEY="2"
```
then modify GRE_VPS_IP and BACKEND_IP to be the additional public IP of the GRE VPS and the IP of the new (or the same) backend server respectively.

⚠️ Also, super importantly, make sure that the iptables -F line on the makeGRE.sh script of the GRE VPS is executed only once by ONLY ONE script. Otherwise the script of each GRE tunnel will keep clearing the iptables rules as they are executed, resulting in an unwanted behaviour.

Now running makeGRE-2.sh on both the backend and the GRE VPS should set this up properly [make sure makeGRE.sh was run first because it has the iptables -F command which clears any unwanted iptables leftovers].

Accessing the additional IP of the GRE VPS should forward the traffic to the same backend server that we set the main GRE tunnel up for. To confirm the setup, run this on the backend server:
```
curl --interface 192.168.168.2 https://icanhazip.com
curl --interface 192.168.169.2 https://icanhazip.com
```
the first command should output the first IP address that we initially set up for the GRE tunnel. and the second command should output the additional IP address that we have just linked to the GRE tunnel.

You can do the same for as many additional IP addresses as you want. Just create makeGRE-3.sh and delGRE-3.sh, and change the 192.168.169 part to something else like 192.168.179
To make the GRE tunnel(s) persistent, create a file at /etc/systemd/system/gre.service with the following content:
```
[Unit]
Description=GREInitService
After=network.target

[Service]
Type=oneshot
ExecStart=/root/makeGRE.sh
ExecStop=/root/delGRE.sh
User=root
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target
```
Then run systemctl daemon-reload, systemctl enable gre.service.

This will:
- make the GRE tunnel(s) automatically get created on the system boot.
- make the management of the GRE tunnel(s) easier. just use systemctl stop gre.service to delete the tunnel(s), and the same for start.
Note that if you have multiple GRE tunnels setup by multiple scripts, it is better to create two scripts called initGRE.sh and deinitGRE.sh

initGRE.sh:
```
#!/bin/bash

/root/makeGRE.sh
/root/makeGRE-2.sh
```
deinitGRE.sh:
```
#!/bin/bash

/root/delGRE-2.sh
/root/delGRE.sh
```
⚠️ Notice how deinitGRE is in the inversed order of initGRE (the last executed makeGRE script is the first executed delGRE script).

Then edit /etc/systemd/system/gre.service to execute the newly created managing scripts instead:
```
ExecStart=/root/initGRE.sh
ExecStop=/root/deinitGRE.sh
```

If you want to make one of the GRE VPS IPs act like the primary IP of the backend server (i.e. all the internet requests on the backend server will see the GRE VPS IP as the public IP of the backend server):

You will need to use these scripts on server B [the backend server] instead of the ones that were shown initially at the top of this whole guide [they are the same scripts with just a few additional commands]:

makeGRE.sh on Server B (the backend server):

#!/bin/bash

# This script is placed on the backend server

#
# Variables
#

GRE_VPS_MAIN_IP="[the main public ip address of the gre vps here]"
GRE_VPS_IP="[the public ip address of the gre vps that you are attempting to forward to the backend server]" # This cannot be the same value of GRE_VPS_MAIN_IP. check the requirements at the top of the guide for more information
BACKEND_IP="[backend server public ip address here]"

GRE_TUNNEL_INTERFACE_NAME="gre1"
GRE_TUNNEL_GREVPS_IP="192.168.168.1"
GRE_TUNNEL_BACKEND_IP="192.168.168.2"
GRE_TUNNEL_RTTABLES_ID="100"
GRE_TUNNEL_RTTABLES_NAME="GRE"
GRE_TUNNEL_KEY="1"

BACKEND_SERVER_MAIN_INTERFACE_NAME="eth0"

# ----------------------------------

GATEWAY_IP=$(ip route show dev $BACKEND_SERVER_MAIN_INTERFACE_NAME | grep default | awk '{print $3}' | awk 'NR==1{print; exit}')

modprobe ip_gre

# https://www.linuxquestions.org/questions/linux-networking-3/how-to-turn-on-keepalive-on-linux-interface-4175541981/#post6320771
sysctl -w net.ipv4.conf.default.accept_local=1
sysctl -w net.ipv4.conf.all.accept_local=1

if ! grep -Fxq "$GRE_TUNNEL_RTTABLES_ID $GRE_TUNNEL_RTTABLES_NAME" /etc/iproute2/rt_tables
then
     echo "$GRE_TUNNEL_RTTABLES_ID $GRE_TUNNEL_RTTABLES_NAME" >> /etc/iproute2/rt_tables
fi

ip tunnel add $GRE_TUNNEL_INTERFACE_NAME mode gre local $BACKEND_IP remote $GRE_VPS_MAIN_IP ttl 255 key $GRE_TUNNEL_KEY
ip link set $GRE_TUNNEL_INTERFACE_NAME up

ip addr add $GRE_TUNNEL_BACKEND_IP/30 dev $GRE_TUNNEL_INTERFACE_NAME
ip addr add $GRE_VPS_IP/32 dev $GRE_TUNNEL_INTERFACE_NAME

ip rule add from $GRE_VPS_IP lookup $GRE_TUNNEL_RTTABLES_NAME
ip route add default via $GRE_TUNNEL_GREVPS_IP table $GRE_TUNNEL_RTTABLES_NAME

iptables -o $GRE_TUNNEL_INTERFACE_NAME -t nat -I POSTROUTING -j SNAT --to-source $GRE_VPS_IP

# dns servers are required otherwise all dns resolutions will fail
echo 'nameserver 1.1.1.1' > /etc/resolv.conf
echo 'nameserver 1.0.0.1' >> /etc/resolv.conf

# finally cut over our routing
# NOTE: this will cut all access to your original BACKEND IP!

ip route add $GRE_VPS_MAIN_IP via $GATEWAY_IP dev $BACKEND_SERVER_MAIN_INTERFACE_NAME onlink
ip route replace default via $GRE_TUNNEL_GREVPS_IP

tc qdisc replace dev $GRE_TUNNEL_INTERFACE_NAME root fq
ip link set $GRE_TUNNEL_INTERFACE_NAME txqueuelen 15000
ethtool -K $GRE_TUNNEL_INTERFACE_NAME gro off gso off tso off

delGRE.sh on Server B (the backend server):

#!/bin/bash

# This script is placed on the backend server

#
# Variables
#

GRE_VPS_MAIN_IP="[the main public ip address of the gre vps here]"
GRE_VPS_IP="[the public ip address of the gre vps that you are attempting to forward to the backend server]" # This cannot be the same value of GRE_VPS_MAIN_IP. check the requirements at the top of the guide for more information

GRE_TUNNEL_INTERFACE_NAME="gre1"
GRE_TUNNEL_GREVPS_IP="192.168.168.1"
GRE_TUNNEL_BACKEND_IP="192.168.168.2"
GRE_TUNNEL_RTTABLES_NAME="GRE"

BACKEND_SERVER_MAIN_INTERFACE_NAME="eth0"

# ----------------------------------

GATEWAY_IP=$(ip route show dev $BACKEND_SERVER_MAIN_INTERFACE_NAME | grep default | awk '{print $3}' | awk 'NR==1{print; exit}')

iptables -t nat -D POSTROUTING -o $GRE_TUNNEL_INTERFACE_NAME -j SNAT --to-source $GRE_VPS_IP

ip route del default
ip route del $GRE_VPS_MAIN_IP via $GATEWAY_IP dev $BACKEND_SERVER_MAIN_INTERFACE_NAME onlink
ip route replace default via $GATEWAY_IP

ip rule del from $GRE_VPS_IP lookup $GRE_TUNNEL_RTTABLES_NAME
ip route del default via $GRE_TUNNEL_GREVPS_IP table $GRE_TUNNEL_RTTABLES_NAME
ip addr del $GRE_VPS_IP/32 dev $GRE_TUNNEL_INTERFACE_NAME
ip addr del $GRE_TUNNEL_BACKEND_IP/30 dev $GRE_TUNNEL_INTERFACE_NAME
ip link set $GRE_TUNNEL_INTERFACE_NAME down
ip tunnel del $GRE_TUNNEL_INTERFACE_NAME

As for the scripts of server A [the GRE VPS], leave them unchanged.

Reboot the GRE VPS (and preferably but not necessarily the backend server[s] too) after setting up or modifying any GRE tunnels to ensure that no unneeded leftovers are there. This really makes a difference most of the time.

⚠️ An important note if you are using BuyVM as your GRE VPS + a DDoS protected IP (or more) from them

Make sure that the main IP address of your BuyVM VPS is the normal non-DDoS protected IP address. You can set the main IP address through the BuyVM Stallion panel.

Also make sure to use that same normal non-DDoS protected IP address as the value of the GRE_VPS_MAIN_IP variable in the scripts.

The main reason we do this is to avoid getting the IP address of our backend server from getting blocked by the BuyVM (Path.net) DDoS protection.

From https://wiki.buyvm.net/doku.php/gre_tunnel:

You will always want to form your GRE with your unfiltered IP address for all GRE tunnels to make sure you don't run into any sort of MTU issues or trigger the DDOS protection.

Also as an additional precaution step, you can go to the DDoS protection panel on your BuyVM Stallion and add a firewall rule like this:

Source IP Address: [the public IP of the backend server]/32
Protocol: ALL (All Protocols)
Action: Allow

oddmario/GRE_TUNNELS.md

UPDATE

Setup a GRE tunnel between two Linux servers

Server A

Server B

This guide covers

Requirements

`makeGRE.sh` on Server A

`delGRE.sh` on Server A

`/etc/sysctl.conf` on Server A

`makeGRE.sh` on Server B

`delGRE.sh` on Server B

Notes

⚠️ An important note if you are using BuyVM as your GRE VPS + a DDoS protected IP (or more) from them

Inspiration

oddmario/GRE_TUNNELS.md

UPDATE

Setup a GRE tunnel between two Linux servers

Server A

Server B

This guide covers

Requirements

makeGRE.sh on Server A

delGRE.sh on Server A

/etc/sysctl.conf on Server A

makeGRE.sh on Server B

delGRE.sh on Server B

Notes

⚠️ An important note if you are using BuyVM as your GRE VPS + a DDoS protected IP (or more) from them

Inspiration

`makeGRE.sh` on Server A

`delGRE.sh` on Server A

`/etc/sysctl.conf` on Server A

`makeGRE.sh` on Server B

`delGRE.sh` on Server B