wireguard, wireguard layer 2, wireguard over TCP

wireguard_layer2.md

Intro

This note describes how to connect two networks/devices/VMs over public network using Wireguard with Layer 2 support (ARP, IPv6 link-local, etc).

This can also be achieved using SSH and its "tap" tunnel, however, it does not provide the same level of latency and bandwidth as full-blown VPN such as Wireguard.

In addition, this note describes how to tunnel Wireguard over TCP connection. This may be of use if you encounter firewall in-between so, for instance, you can use TCP port 443 only.

Objective

Let's draw a network diagram of what we would like to have in as a result.

 Host on private LAN                                         Host on the Internet
 +---------------------------------+                       +-------------------------------+
 |   brtun bridge                  |                       |                  brtun bridge |
 | +-------------+                 |                       |                 +--------+    |
 | | ethX gretun |<->wg0<->udp2raw | <-Internet, TCP 443-> | udp2raw<->wg0<->| gretun |    |
 | +-------------+                 |                       |                 +--------+    |
 +---------------------------------+                       +-------------------------------+
 
 brtun: 192.168.0.200/24                                    brtun: 192.168.0.50/24
 wg0: 12.12.12.2/24                                         wg0: 12.12.12.1/24
 gretun: 12.12.12.2->12.12.12.1                             gretun: 12.12.12.1->12.12.12.2
 

Setting things up

Distinguishing between server and client here is based on the answer to the question: who can connect to Wireguard listener socket.

Server

Generating a key:

wg genkey | tee wgkeyprivs | wg pubkey > wgkeypubs

Creating a Wireguard interface, setting the private key and a unique private IPv4 address:

ip l a wg0 type wireguard
wg set wg0 private-key ./wgkeyprivs
ip a a 12.12.12.1/24 dev wg0

In case we want Wireguard over TCP, we have to decrease MTU:

ip l set dev wg0 mtu 1200

Configuring peer's public key and setting interface up (you can restrict allowed-ips to the range you want):

ip l set dev wg0 up
wg set wg0 listen-port 51820 peer $(cat wgkeypubc) allowed-ips 0.0.0.0/0

Creating GRETAP interface which will provide Layer 2 over our Wireguard tunnel:

ip l a gretun type gretap local 12.12.12.1 remote 12.12.12.2
ip l s gretun up

Configuring a bridge which includes GRETAP interface. This step is in fact optional, but it allows you some flexibility: you can assign an IP address to this bridge interface and/or you can add additional interfaces to this bridge.

ip l add name brtun type bridge
ip l set dev brtun up
ip l set gretun master brtun

Let's assign an IP address from the target network to this bridge:

sudo ip a a 192.168.0.51/24 dev brtun

If we want Wireguard over TCP, launch UDP2RAW listener which will do its magic. You can get UDP2RAW from this Git repo: https://github.com/wangyu-/udp2raw-tunnel/releases/

./udp2raw_amd64 -s -l YOUR_PUBLIC_IP:443 -r 127.0.0.1:51820 -a --fix-gro

Client

Client is very similar to the server described above.

Generating a key:

wg genkey | tee wgkeyprivc | wg pubkey > wgkeypubc

Creating a Wireguard interface, setting the private key and a unique private IPv4 address:

ip l a wg0 type wireguard
wg set wg0 private-key ./wgkeyprivc
ip a a 12.12.12.2/24 dev wg0

In case we want Wireguard over TCP, we have to decrease MTU:

ip l set dev wg0 mtu 1200

Configuring peer's public key, endpoint address and setting interface up (you can restrict allowed-ips to the range you want):

ip l set dev wg0 up
wg set wg0 listen-port 51820 peer $(cat wgkeypubs) allowed-ips 0.0.0.0/0 endpoint YOUR_PUBLIC_IP:51820 persistent-keepalive 15

In case you want Wireguard over TCP, set endpoint address to a localhost interface:

wg set wg0 listen-port 51820 peer $(cat wgkeypubs) allowed-ips 0.0.0.0/0 endpoint 127.0.0.1:7777 persistent-keepalive 15

Creating GRETAP interface which will provide Layer 2 over our Wireguard tunnel:

ip l a gretun type gretap local 12.12.12.2 remote 12.12.12.1
ip l s gretun up

Configuring a bridge which includes GRETAP interface.

ip l add name brtun type bridge
ip l set dev brtun up
ip l set gretun master brtun

Let's add a physical interface belonging to a LAN network to the bridge and start DHCP client:

ip l set ethX master brtun
dhcpcd ethX

If we want Wireguard over TCP, launch UDP2RAW listener on the localhost interface which connects to the corresponding public listener.

./udp2raw_amd64 -c -l 127.0.0.1:7777 -r YOUR_PUBLIC_IP:443 -a --fix-gro

Results

That's it. Now you will have Layer 2 connectivity between two separated machines over Wireguard VPN (optionally, over TCP). Enjoy your ARP and IPv6 link-local experience.

References

• https://notes.superlogical.ch/pages/note_wg/nolayer2/
• https://lafibre.info/vpn/tunnel-gre-sur-wireguard/&prev=search&pto=aue

Thanks for this article.
I was looking for a fastest option to bridge two locations with same LAN network.
Wireguard plus GRETAP and bridging seems to be the best option for this. I followed this article and with the help, I am able to
connect two hosts to reach each other with LAN IP addresses . I.e with this method we can connect two systems with a bridge over Wireguard and GRETAP. But when try to reach other systems than the Wireguard host, it seems not working.

On a boarder way I am able to see that Wireguard is doing some mischief with 'Allowed IPS' option.
Being both side same LAN I am forced to use 'Allowed IPS' same network. Say for example if you
are planning to extend 192.168.5.0/24 with this solution, we need to add Allowed IPs = 192.168.5.0/24 on both side.
in addition to the WG interface IPs. This seems creating problem as other systems in the same network can't reach
Wireguard system LAN ip address as Wireguard forwarding that packets to peer system. So once Wireguard is up,
the Wireguard systems are not ping-able from the LAN network on each side, because Wireguard seems to forward that
packets to the peer host instead of responding to ping.

Wondering any workaround for this problem ?

Hi @SandeepAthiyarath,

With AllowedIPs and routing it is all about careful and accurate planning, not to break the stick you are sitting on. :) So yes, with some configurations you indeed can block you own traffic. I would suggest to exclude VPN server's IP address from internal networks. Or even come up with private segment just for VPN segment, depends. So you can have IP address to connect to outside of AllowedIPs range.
BTW, you can have many networks inside AllowedIPs option.

Another scenario (but may be weird) is to have a masquerading of some sort which will hide the network segment behind itself. In this case Wireguard will receive all packets from a single address.

Nevertheless, I am just producing some thoughts. :) I never tried to build production/office networks with such configuration. However, there are even paid products offering such, so it should be possible.

Hmm. Thank you. This is valuable inputs.
In any case we are not going to use duplicate IP s on both side.
I was expecting a magic solution. Now I understand it wont work like that :)
Still I never came across a commercial solution to bridge LANs. If you have any reference I am eager to look at it.
Thank you for your valuable article and comment @zOrg1331.

HI @zOrg1331
I am clueless here after connecting with other systems in the LAN. I managed to use Allowed IPs with /32 and expecting it is working. As mentioned in my previous comment I am able to communicate with LAN IPs between the Wireguard hosts.
Problem starts after I add LAN side interface to bridge. This looks weird. My LAN side IP is 192.168.17.1 on enp2s0 interface. I have a
system with 192.168.17.5 connected with a switch to enp2s0 of Wireguard host.
The moment I add enp2s0 to the bridge these systems stops communicating each other. No Ping. No arp info. If I remove enp2s0 from the
bridge with nomaster option, then they again start communicating each other. I am clueless here.

root@srv1 fcoos]# bridge l
3: enp2s0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 master brtun state forwarding priority 32 cost 4
12: gretun@NONE: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1382 master brtun state forwarding priority 32 cost 100
If I run
root@srv1 fcoos]# ip l set enp2s0 nomaster
[root@srv1 fcoos]# ping 192.168.17.5
PING 192.168.17.5 (192.168.17.5) 56(84) bytes of data.
64 bytes from 192.168.17.5: icmp_seq=1 ttl=64 time=0.166 ms

But If I add again to the bridge
oot@srv1 fcoos]# ip l set enp2s0 master brtun up
[root@srv1 fcoos]# ping 192.168.17.5
PING 192.168.17.5 (192.168.17.5) 56(84) bytes of data.
From 192.168.17.1 icmp_seq=1 Destination Host Unreachable

I was expecting the reverse of this. Thinking I am doing something wrong here. But what is that ? Please note that I need to bridge two LANS not two hosts. I have bridged LAN with OpenVPN tap. But performance is not as expected. I am hopeful with wire-guard. But :). Need help

Hello @zOrg1331, @SandeepAthiyarath

@zOrg1331 I asked you some questions a couple of weeks ago and I'm now reporting back to you. @SandeepAthiyarath I hope my findings can be useful for you.

I currently have a Wireguard + GRETAP system working successfully and I'm going to document here what I did. My objective was to connect 2 sites -- one on each side of the Atlantic ocean -- as if they were on a single LAN. A key requirement for me was that broadcast traffic be seen on both sides. There was some trial and error to get this to work, but now that it's working, I'm getting at least 4x the performance I used to get with OpenVPN TAP, and significantly better than what I get with Zerotier. I'm getting 200 Mbps in both directions between the two sites (my Internet speed in North America is the limiting factor). Here is a step-by-step explanation of what I did.

BACKGROUND

In my network, both sides are on the same subnet -- 192.168.15.0/24. Each side has its own DHCP server. However, the addresses given out by Side A's router (192.168.15.1) go from .100-.169 and by Side B's router (192.168.15.2) from .170-.254. Using ebtables, I block DCHP requests from going over the VPN, so each side only sees its local DHCP server (I explain this at the bottom). I have a number of static IP addresses in use on either side but they are all in the 10-99 range.

I have dedicated Wireguard gateway servers on each side. The physical hardware for each server is a $160 Beelink Intel Quad-core with 1 x 1 Gb Ethernet port. The servers are running Ubuntu 22.04 LTS and they are using the default "Netplan" system for configuring everything related to networking. I realize most distributions don't use Netplan, but I find Netplan very sensible and easy to understand because it puts all network configuration in one single file (although it's a little fussy when it comes to proper indenting).

One other thing. I use UDP port 52001 for Wireguard on both sides, so on each side I have to forward that port from my router to the Wireguard server. So, here's how I set everything up (with much thanks to @zOrg1331 for helping me get started). My instructions assume you have already installed the Wireguard tools packages.

STEP ONE -- GENERATE WIREGUARD KEYS

NOTE: The keys used below are not my keys. I found them in some other random Wireguard example.

Side A -- North America

wg genkey | tee wgkeyprivs | wg pubkey > wgkeypubs
cat wgkeyprivs  SF8baNt/6uS8r6rAB7XNUyWrjo89Tf/q93Y1oG2YRWo=   
cat wgkeypubs  mnhaI1xPKVC/yKUmn77i3nuKtVfxnkg9ojIXd5CBn0E=

Side B -- Europe

wg genkey | tee wgkeyprivs | wg pubkey > wgkeypubs
cat wgkeyprivs  iETL+NzI0VtxxCz62ZtG4N9ELoq1XLD6PmUb7fFir0g=
cat wgkeypubs  bGgYeUjhy+LRKMPL8lgcXlxKlpUJGuqa3mMgX9fw+ng=

In the next step, we will copy and paste Side A's public key into Side B's Wireguard configurationso that Side B can gain access to Side A. And vice versa.

STEP TWO -- CONFIGURE NETWORKING FOR WIREGUARD

On Side A -- North America -- modify /etc/netplan/00-installer-config.yaml and apply changes.

I have only one physical Ethernet port (enp1s0) on the Beelink computer that I use for Wireguard here, and I want it to have a static address of 192.168.15.20 and I want that to be part of a bridge br0. I want to set the IP address and default gateway (the LAN address of my American router) ONLY on the bridge interface. The physical interface should NOT have an IP Address. I also want to create the Wireguard tunnel so that it will have the address 172.16.0.1/24 and automatically connects to the European side.

Contents of /etc/netplan/00-installer-config.yaml

# This is the network config written by 'subiquity'
network:        
  ethernets:
    enp1s0:
      dhcp4: no

  bridges:
    br0:
      dhcp4: no
      addresses: [192.168.15.20/24]
      nameservers:
        addresses: [192.168.15.1]
      routes:
        - to: default
          via: 192.168.15.1
      interfaces:
        - enp1s0
      parameters:
        stp: false

  tunnels:
    wg0:
      mode: wireguard
      port: 52001
      key: SF8baNt/6uS8r6rAB7XNUyWrjo89Tf/q93Y1oG2YRWo=   
      addresses:
        - 172.16.0.1/24
      peers:
        - allowed-ips: [172.16.0.0/24]
          endpoint: [EUROPEAN SIDE DOMAIN NAME]:52001
          keys:
            public: bGgYeUjhy+LRKMPL8lgcXlxKlpUJGuqa3mMgX9fw+ng=
  version: 2

To apply, run the command "sudo netplan apply"

On Side B -- Europe -- modify /etc/netplan/00-installer-config.yaml and apply changes.

I have only one physical Ethernet port (enp1s0) on the Beelink computer that I use for Wireguard here, and I want it to have a static address 192.168.15.21 and I want that to be part of a bridge br0. I want to set the IP address and default gateway (my European router) only on the bridge. I also want to create the Wireguard tunnel so that it will have the address 172.16.0.2/24 and automatically connect to the European side.

Contents of /etc/netplan/00-installer-config.yaml

# This is the network config written by 'subiquity'
network:        
  ethernets:
    enp1s0:
      dhcp4: no

  bridges:
    br0:
      dhcp4: no
      addresses: [192.168.15.21/24]
      nameservers:
        addresses: [192.168.15.2]
      routes:
        - to: default
          via: 192.168.15.2
      interfaces:
        - enp1s0
      parameters:
        stp: false

  tunnels:
    wg0:
      mode: wireguard
      port: 52001
      key: iETL+NzI0VtxxCz62ZtG4N9ELoq1XLD6PmUb7fFir0g= 
      addresses:
        - 172.16.0.2/24
      peers:
        - allowed-ips: [172.16.0.0/24]
          endpoint: [NORTH AMERICAN SIDE DOMAIN NAME]:52001
          keys:
            public: mnhaI1xPKVC/yKUmn77i3nuKtVfxnkg9ojIXd5CBn0E=
  version: 2

To apply, run the command "sudo netplan apply"

STEP 3 CONFIRM THAT WIREGUARD IS RUNNING AND CONNECTED TO THE PEER ON THE OTHER SIDE

On the European side if I run "sudo wg show" I see:

interface: wg0
  public key: mnhaI1xPKVC/yKUmn77i3nuKtVfxnkg9ojIXd5CBn0E=
  private key: (hidden)
  listening port: 52001

peer: bGgYeUjhy+LRKMPL8lgcXlxKlpUJGuqa3mMgX9fw+ng=
  endpoint: [IP ADDRESS OF NORTH AMERICAN SIDE]:52001
  allowed ips: 172.16.0.0/24
  latest handshake: 1 minute, 8 seconds ago
  transfer: 13.27 GiB received, 9.44 GiB sent

Once Wireguard is running on both sides, I should be able to ping from one Wireguard server to the other across the Atlantic. For example, from 172.16.0.1 I get:
vpnadmin@andyvpntorre:/etc/netplan$ ping 172.16.0.2
PING 172.16.0.2 (172.16.0.2) 56(84) bytes of data.
64 bytes from 172.16.0.2: icmp_seq=1 ttl=64 time=130 ms
64 bytes from 172.16.0.2: icmp_seq=2 ttl=64 time=126 ms
64 bytes from 172.16.0.2: icmp_seq=3 ttl=64 time=134 ms
64 bytes from 172.16.0.2: icmp_seq=4 ttl=64 time=138 ms

VERY IMPORTANT! Note, at this point, from the Wireguard server at 192.168.15.20 I cannot ping the Wireguard server at 192.168.15.21. In other words, the Wireguard servers don't know anything about how to cross the Atlantic on the 192.168.15.0/24 network. But they soon will.

STEP 4 -- CREATE GRETAP INTERFACES AND ADD THEM TO THE BRIDGE br0

I haven't yet tried putting this in my netplan configuration (I will soon and I'll come back here and edit this post) so after I have Wireguard running properly on each side and connected to the other side, I now create the GRETAP interface so that my Ethernet packets will flow from any computer on Side A to any computer on Side B (including from 192.168.15.20 to 192.168.15.21).

On Side A run the following commands:

sudo ip link add gretun type gretap local 172.16.0.1 remote 172.16.0.2 ignore-df nopmtudisc

This creates a virtual interface called "gretun" of the type "gretap". Packets will go from local IP address 172.16.0.1 to remote address 172.16.0.2. Note the addition of the two options "ignore-df" and "nopmtudisc". Without these, I found that packets larger than 1354 bytes could not get through the tunnel. For instance, web pages won't load across the tunnel. This is due to large packets not getting fragmented into pieces that are small enough to go through the tunnel. The large packets just get dropped without those options.

ip link set gretun up

This brings up the gretun interface

ip link set gretun master br0

This adds the gretun interface to my bridge "br0"

ip link set gretun mtu 1500

The default MTU that gets created for the GRETAP interface that we named "gretun" is too low and even with the "ignore-df" and "nopmtudisc" options. Setting the MTU to 1500 ensures that all normal network packets will get through.

On Side B, do the same thing except swap the IP addresses:

sudo ip link add gretun type gretap local 172.16.0.2 remote 172.16.0.1 ignore-df nopmtudisc
ip link set gretun up
ip link set gretun master br0
ip link set gretun mtu 1500

Assuming you have bridge-utils installed, you should see the following when you run the command "brctl show"

bridge name	bridge id		STP enabled	interfaces
br0		8000.5e085b91133c	no		enp1s0
						        gretun

STEP 5 -- BLOCK DHCP REQUESTS FROM CROSSING OVER THE VPN

On the Wireguard server on each side, I run the following commands:

ebtables -A INPUT --in-interface gretun --protocol ipv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP
ebtables -A INPUT --in-interface gretun --protocol ipv4 --ip-protocol udp --ip-source-port 67:68 -j DROP
ebtables -A FORWARD --out-interface gretun --protocol ipv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP
ebtables -A FORWARD --out-interface gretun --protocol ipv4 --ip-protocol udp --ip-source-port 67:68 -j DROP

I will eventually put this into a script that runs a minute or so after the Wireguard-GRETAP bridge is up and connected.

FINAL NOTES

• To make Wireguard + GRETAP work, you DO NOT need to enable IPV4 ip_forwarding. This works because the interface we named "gretun" (of type "gretap") is part of the bridge br0.
• You do need to have a Domain Name or Dynamic DNS name for each side, or static public IP addresses, so that the two endpoints can automatically find each other and connect.
• There is no routing involved when you put together Wireguard + GRETAP. This is just like having two Ethernet switches on your network, only there's a very long WAN cable between them (with both sides on the same subnet).
• When a Wireguard interface gets created, it gets a default MTU of 1420. Changing this MTU to a bigger number doesn't seem to make any difference, nor does it solve the problem that only gets solved by adding the options "ignore-df nopmtudisc" to the command that creates the GRETAP interface.
• I'm still considering whether I need to add a "persistent keepalive" option to my Wireguard configuration

Excellent update @AndrewL733 . Thank you. I noticed one main difference is that you use single network card system with NATed from router.
Ideally it should not make difference with two cards as in my case. Another important thing I noticed that you use Allowd IPs complete sub-net.
I was thinking that this wont work . I am going to try options and update here soon.

Hi!
I want to say thank you to all of you and the final guide of @AndrewL733
I was trying to achieve this to protect my home network behind a VPS with DDOS protection.
Because my home ISP don't allow me to open ports, WireGuard was the only approach.

In my case the IP of the VPS it's the public one, like a modem in bridge mode.
In that case I put two addreses with comma, one for the public IP of the VPS, and another for the VLAN of my network.
The same for the addresses in the nameservers.

In this escenario I enable in the VPS "the server" sysctl -w net.ipv4.ip_forward=1 because the traffic from the public IP must be NATed to a computer inside my home VLAN, and need to enable a iptables -t nat -A POSTROUTING -o br0 -j MASQUERADE to expose the br0 interface to internet.
Finally in the Windows computer I have in my home need to add like a secondary gateway, in this scenario the public IP of the VPS.

And works awesome!

I add the persistent keepalive because happens to me some incoming packets mostly in UDP don't arrive to the computer behind the NAT, and some services that are monitored with UDP ports believe it's down, but with the persistent works great.

The part of the greetap interfaces I made it with a script in the startup of the ubuntu server in each side, then use in all cases aliases because in the home network the public IP can change time to time.

And Finally install UFW in the server side because have the public IP to BLOCK all except what I need to nat to my house, to avoid any possible issue.

Again, thanks to everyone for this process and sharing it!

now it should be possible to use vrrp over wireguard.

@zOrg1331 Thanks for your reply. Can you explain whether the configuration you have described would create a site-to-site bridge? The answer seems to be “no" but maybe I’m missing something. My ultimate goal is to create a site-to-site Wireguard VPN with layer 2 support, such that all computers at Site A can communicate with all computers at Site B, and so that all broadcast traffic goes over the VPN. I am currently able to achieve this with OpenVPN TAP and also with ZeroTier (about 1.3x the performance as OpenVPN). It would seem very useful to be able to use Wireguard for the same purpose.

Well, as for me, the whole purpose of this exercise was to link two networks on L2 level by using Wireguard.

Side note: I used it for a quite non-standard purpose. During a penetration test I had to establish a tunnel towards a corporate network via a corporate firewall and have L2 communications at the same time. By following the described method I had transparent (for userspace tools) L2 communications between a Wireguard client (remote VM) and the remote network. So the only difference between site-to-site VPN was that I did not have network with multiple hosts on one side.

Once you have GRE L2 tunnel working over Wireguard, the rest is up to you and your setup.

but what if you wanna connect multiple networks together?

Спасибо за хорошую инструкцию. Нашел ошибку в своей конфигурации: на центральном сервере, соединяющем 2 wireguard был настроен маскарад

please tell me why the solution does not work?:

ip link add vx0 type vxlan id 200 group 239.1.1.1 dev wg0 dstport 4789
ip link set vx0 master br0
ip link set vx0 up

Hi all,

Thank you very much for your guides, they are really helpful.

@AndrewL733, how do you do the tunnel permanent? I'm loosing the setup after rebootin:

sudo ip link add gretun type gretap local 172.16.0.2 remote 172.16.0.1 ignore-df nopmtudisc
ip link set gretun up
ip link set gretun master br0
ip link set gretun mtu 1500

Thanks in advance!
Best regards

Hi @pmagro just login with sudo su - to be root, create a Start.sh in the /root folder then type chmod a+x /root/Start.sh to set the permission
Then put that code without the sudo in front in the Start.sh file, save it.
Then type crontab -e
Add at the end:

@reboot /root/Start.sh

Save it.

Done. Next time you start up your system will add that code.
Good Luck.

Superb, @geckotdf, it worked, thanks!

Can I ask another question? What would be the public IP address I will be detected with when accessing internet from each of the sites?

Thanks in advance!

Best regards
Pablo

In the other sites will have the public IP that have the gateway in that computers.
That not going to change, this expose layer 2.
Only if you do what I post a few months ago, in the server:

sysctl -w net.ipv4.ip_forward=1
iptables -t nat -A POSTROUTING -o br0 -j MASQUERADE

Then, in the remote computer set in the gateway the public IP of the server.
But this only will work if the server have the public IP in the interface like a VPS or dedicated server.

For example I have more or less a setup like this to expose my services behind a server and avoid exposing my real IP, and prevent DDOS attacks in my services, similar to a Proxy, but allow UDP and TCP Traffic.

192.168.15.0/24

@AndrewL733 great guide.

I double checked everything and was able to ping 172.16.0.2 and 172.16.0.1 from both sides but traffic from my 192.168.x network is not passing through the bridge. Any suggestions?

Hi @jonfatino, I can confirm that following Andrew's guide I made it work. Trying double checking everything or start from scratch if needed, but following it step by step, it works.

@AndrewL733 , Great explanation. But I did notice drastic difference when I bumped up the MTU of wg interface to 1550. I'm testing in a lab environment were all the devices have 10GBPS NIC. When wg0 interface is left to default of 1420 I got around 2GBPS of throughput. But after changing it to 1550 I'm getting throughput around 4GBPS.https://www.reddit.com/r/networking/comments/19ecn6v/comment/kjeetzs/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button redditor put dose similar with VXLAN and wg and has got 870+ on a 1GBPS link. I'm expecting atlease a throughput of 6GBPS with this setup. I hope I'm not missing anything.

Thanks

@Gaitonde007
Im not an expert but in general lines a big MTU can cause packet fragmentation and a little one to much overload.
So... probably at that fast speeds put a more tiny MTU like 1420 make load in the network and you are only able to get 2 GBPS, and a bigger one, allow to get 4 GBPS.
Take in mind the kind of traffic it's traveling too, it's not the same UDP packets than for example TCP.

Reading about what Andrew test, the default 1420 don't work very well like he said for navigation, so probably after testing get that 1500 it's the lowest working MTU quote:

The default MTU that gets created for the GRETAP interface that we named "gretun" is too low and even with the "ignore-df" and "nopmtudisc" options. Setting the MTU to 1500 ensures that all normal network packets will get through.

So... if you need more throughput can test changing the MTU, at the end of the day need to be set based on the network equipment and topology.
But it's a general line opinion probably a person with more experience can tell.

@AndrewL733 thank you very much for the tutorial, I have finally been able to connect my two homes!

I have the following problem, I'm wondering if someone can help me:

• In both homes A and B I have a router that broadcasts IPTV multicast content.
• I have multicast receivers in both homes.
• I want the multicast receiver in apartment B to receive content from apartment A.
• Receivers and routers use IGMPv2.
• On receiver B I have set a static IP with gateway to router A

I can't get it to work, in the firewalls of the two wireguard servers I have allowed all incoming/outgoing traffic to 224.0.0.0/4

for fake tcp,try phantun or mimc(debian 12 or kernel 6.1+),faster if you want.And I prefer use vxlan via wg.

@AndrewL733 - Your guide is really good, and it works. :) However, I do have an issue, and I'm a bit lost.

I have tried it between 2 data centers. When I install 2 Windows servers, one on each site (Site A and Site B), they do timeout when pinging each other. I did manage to get a speed test with over a Gigabit connection through the tunnel, but after the test, the connection dies. Then I need to reboot the WireGuard "switches" and wait 3-10 minutes to have a somewhat ok connection again. I have tried to change the MTU and also did an MTU test through the tunnel, but with no luck.

I can see some have commented on the high throughput and suggested changing the MTU. Has anyone figured it out? :)

The point of this for me is because I need to migrate some servers from one data center to another, and if I could make a big "switch", it would be nice to just take parts of the environment at a time.

I will paste my configuration down here, for Site A and Site B, and I hope you, or maybe someone else, have experienced the same issue and maybe have a solution.

Thank you for your time :)

Site A

#Ubuntu 22.04 LTS
#Remember to have Promiscuous mode, MAC address changes, and Forged transmits enabled on the interfaces.
#Change to root.
sudo su -
apt purge snapd -y;apt update -y ; apt full-upgrade -y;apt install wireguard bridge-utils openvswitch-switch-dpdk traceroute net-tools -y
reboot now
#Login again
sudo su -
wg genkey | tee wgkeyprivs | wg pubkey > wgkeypubs
cat wgkeyprivs
cat wgkeypubs
Private: SiteAPrivateKey
Public: SiteAPublicKey

nano /etc/netplan/00-installer-config.yaml
#Change the netplan configuration. Modify it for the respective interfaces, IPs, and WireGuard keys.
# This is the network config written by 'subiquity'
network:
  ethernets:
    #Uplink/switching interface 10.76.0.0/24
    ens192:
      dhcp4: false
      dhcp6: false

  bridges:
    br1:
      dhcp4: false
      dhcp6: false
      #IP to Wireguard VM
      addresses: [10.76.1.10/24]
      #DNS - This can be changed to the Domain Controller if needed
      nameservers:
        addresses:
          - 1.1.1.1
          - 1.0.0.1
      routes:
        - to: default
          via: 10.76.1.1
      interfaces:
        - ens192
      parameters:
        stp: false

  tunnels:
    wg0:
      mode: wireguard
      port: 51820
      #Site A Privatekey
      key: SiteAPrivateKey
      addresses:
        - 172.16.0.1/24
      peers:
        - allowed-ips: [172.16.0.0/24]
          endpoint: SiteB_WAN:51820
          #Peers public key
          keys:
            public: SiteBPublicKey
          keepalive: 90
  version: 2

#Activate Layer2
#Create bash script, with execute permission.
nano ~/gretun_cmd
  #!/bin/bash
  ip link add gretun1 type gretap local 172.16.0.1 remote 172.16.0.2 ignore-df nopmtudisc
  ip link set gretun1 up
  ip link set gretun1 master br1
  ip link set gretun1 mtu 1500
chmod u+rwx,g+r-wx,o-rwx ~/gretun_cmd
bash ~/gretun_cmd

#Create bash script, to Block DHCP Req over WireGuard VPN. 
nano ~/block_dhcp_cmd
  #!/bin/bash
  ebtables -A INPUT --in-interface gretun1 --protocol ipv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP
  ebtables -A INPUT --in-interface gretun1 --protocol ipv4 --ip-protocol udp --ip-source-port 67:68 -j DROP
  ebtables -A FORWARD --out-interface gretun1 --protocol ipv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP
  ebtables -A FORWARD --out-interface gretun1 --protocol ipv4 --ip-protocol udp --ip-source-port 67:68 -j DROP
chmod u+rwx,g+r-wx,o-rwx ~/block_dhcp_cmd
bash ~/block_dhcp_cmd

#Now Layer2 is active

Site B

#Ubuntu 22.04 LTS
#Remember to have Promiscuous mode, MAC address changes, and Forged transmits enabled on the interfaces.
#Change to root.
sudo su -
apt purge snapd -y;apt update -y ; apt full-upgrade -y;apt install wireguard bridge-utils openvswitch-switch-dpdk traceroute net-tools -y
reboot now
#Login again
sudo su -
wg genkey | tee wgkeyprivs | wg pubkey > wgkeypubs
cat wgkeyprivs
cat wgkeypubs
Private: SiteBPrivateKey
Public: SiteBPublicKey

nano /etc/netplan/00-installer-config.yaml
#Change the netplan configuration. Modify it for the respective interfaces, IPs, and WireGuard keys.
# This is the network config written by 'subiquity'
network:
  ethernets:
    #Uplink interface 10.76.2.0/24. This allows connection through WireGuard, with traffic passing through this interface.
    ens7:
      dhcp4: true
      dhcp6: false

    #Switching interface for bridge br1 10.76.1.0/24 
    #This is a dummy interface, BUT STILL REQUIRING Promiscuous mode, MAC address changes, and Forged transmits TO BE ACTIVE ON THE INTERFACE.
    #No gateway should be configured, as it goes through the WireGuard tunnel.
    ens3:
      dhcp4: false
      dhcp6: false

  bridges:
    br1:
      dhcp4: false
      dhcp6: false
      addresses: [10.76.1.11/24]
      #DNS - This can be changed to the Domain Controller if needed
      nameservers:
        addresses:
          - 1.1.1.1
          - 1.0.0.1
      #No routing needs to be configured; everything is routed through the WireGuard tunnel.
#      routes:
#        - to: default
#          via: 10.76.1.1
      interfaces:
        - ens3
      parameters:
        stp: false

  tunnels:
    wg0:
      mode: wireguard
      port: 51820
      #Site B Private key
      key: SiteBPrivateKey
      addresses:
        - 172.16.0.2/24
      peers:
        - allowed-ips: [172.16.0.0/24]
          endpoint: SiteA_WAN:51820
          #Peers public key
          keys:
            public: SiteAPublicKey
          keepalive: 90
  version: 2

#Activate Layer2
#Create bash script, with execute permission.
nano ~/gretun_cmd
  #!/bin/bash
  ip link add gretun1 type gretap local 172.16.0.2 remote 172.16.0.1 ignore-df nopmtudisc
  ip link set gretun1 up
  ip link set gretun1 master br1
  ip link set gretun1 mtu 1500
chmod u+rwx,g+r-wx,o-rwx ~/gretun_cmd
bash ~/gretun_cmd

#Create bash script, to Block DHCP Req over WireGuard VPN.
nano ~/block_dhcp_cmd
  #!/bin/bash
  ebtables -A INPUT --in-interface gretun1 --protocol ipv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP
  ebtables -A INPUT --in-interface gretun1 --protocol ipv4 --ip-protocol udp --ip-source-port 67:68 -j DROP
  ebtables -A FORWARD --out-interface gretun1 --protocol ipv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP
  ebtables -A FORWARD --out-interface gretun1 --protocol ipv4 --ip-protocol udp --ip-source-port 67:68 -j DROP
chmod u+rwx,g+r-wx,o-rwx ~/block_dhcp_cmd
bash ~/block_dhcp_cmd

#Now Layer2 is active

Can someone help me out here. This is what I am trying to accomplish.

Server in data center

• Linux/Ubuntu single interface public ip. I can add more interfaces/vlans if needed but would like to keep this as simple as possible.

Server at home/lab (proxmox/vmware)

• Home network is like others a public ip -> router/gateway (ubuquiti unifi) -> nat/lan
• VM with two interfaces. eth0 is 192.168.40.X/24 and has outbound network access to connect to the server at DC
• Eth1 is a blank interface that I want to bridge to a vswitch. From that vswitch I want to spin up other vms and use some of the public ips from my server network.

So once l2/wg is setup. Ideally all I would need to do is spin up a fresh server. Select that bridged vswitch and could use the public ips/gw's as if the vm was running in the data center. Obviously the traffic is going to be backhauled through wg/l2/vpn.

Is this possible? Can any assist me with a modified guide? Is wireguard the best tool for this? Gre tunnel instead? I am open to suggestions. Please see below picture example.

Image backup url https://imgur.com/a/KJc6kdM

Can someone help me out here. This is what I am trying to accomplish.

Server in data center

• Linux/Ubuntu single interface public ip. I can add more interfaces/vlans if needed but would like to keep this as simple as possible.

Server at home/lab (proxmox/vmware)

• Home network is like others a public ip -> router/gateway (ubuquiti unifi) -> nat/lan
• VM with two interfaces. eth0 is 192.168.40.X/24 and has outbound network access to connect to the server at DC
• Eth1 is a blank interface that I want to bridge to a vswitch. From that vswitch I want to spin up other vms and use some of the public ips from my server network.

So once l2/wg is setup. Ideally all I would need to do is spin up a fresh server. Select that bridged vswitch and could use the public ips/gw's as if the vm was running in the data center. Obviously the traffic is going to be backhauled through wg/l2/vpn.

Is this possible? Can any assist me with a modified guide? Is wireguard the best tool for this? Gre tunnel instead? I am open to suggestions. Please see below picture example.

Image backup url https://imgur.com/a/KJc6kdM

@JonEhh1337
It sounds like my config, try give that a look, it might help you :)
(Check the comment above)

For the datacenter part, I would look at my Site A, and at Home give Site B a look ;)

Hello @zOrg1331, @SandeepAthiyarath

@zOrg1331 I asked you some questions a couple of weeks ago and I'm now reporting back to you. @SandeepAthiyarath I hope my findings can be useful for you.

I currently have a Wireguard + GRETAP system working successfully and I'm going to document here what I did. .......

Hi, thanks for your work, I followed your guidelines and it is working properly. A question: why are only the two servers that make the tunnel not reachable from network trunks on the opposite side?
Thanks again.
Marco

Is such a tunnel a full-fledged tunnel that openvpn provides in tap mode? What is the speed like? How much slower will it be than a standard wireguard connection? Has anyone tried to make a tunnel on an openwrt client? If you have server A and client B, can the client receive a DHCP address from server A (as in the case of openvpn), or does it have to set the address itself?