Home Server setup: Raspberry PI on Internet via reverse SSH tunnel

home-server.md

Raspberry Pi on Internet via reverse SSH tunnel

HackerNews discussed this with many alternative solutions: https://news.ycombinator.com/item?id=24893615

I already have my own domain name: mydomain.com. I wanted to be able to run some webapps on my Raspberry Pi 4B running perpetually at home in headless mode (just needs 5W power and wireless internet). I wanted to be able to access these apps from public Internet. Dynamic DNS wasn't an option because my ISP blocks all incoming traffic. ngrok would work but the free plan is too restrictive.

I bought a cheap 2GB RAM, 20GB disk VM + a 25GB volume on Hetzner for about 4 EUR/month. Hetzner gave me a static IP for it. I haven't purchased a floating IP yet.

I created a subdomain u1.mydomain.com with its A record pointing to the above static IP address.

Then I created two CNAME records, pointing to the above subdomain: home.mydomain.com and cloud.mydomain.com.

I disabled nginx on the server and installed Caddy instead. These are the contents of my /etc/caddy/Caddyfile:

{
  email [email protected]
}

home.mydomain.com {
  # These are reverse-proxied to port 10000+n which are SSH
  # tunneled into the raspberrypi at my home
  reverse_proxy localhost:10080
}

cloud.mydomain.com {
  # These are served locally but with automatic HTTPS
  root * /usr/share/caddy
  file_server
}

I set GatewayPorts clientspecified in my /etc/ssh/sshd_config on the server. This is needed so that the client (RaspberryPi) can specify which ports to enabled reverse tunneling on.

I enabled the Ubuntu firewall on this server and allowed incoming traffic on port 80, and 443. For eg: sudo ufw allow 80

Now, on my Raspberry Pi at home, I created a reverse SSH tunnel to this Hetzner VM with: ssh -N -T -R 10080:localhost:80 [email protected]

And just like that, my site running on port 80 on the Pi is now accessible at https://home.mydomain.com.

Once we're done testing, we can add the -f option so that this tunnel runs in the background. To be able to recreate the tunnel (perhaps via crontab), I put this in a bash script:

#!/bin/bash
createTunnel() {
  /usr/bin/ssh -f -N -T -R 10080:localhost:80 [email protected]
  if [[ $? -eq 0 ]]; then
    echo Tunnel to Hetzner created successfully
  else
    echo An error occurred creating a tunnel to hetzner. RC was $?
  fi
}
/bin/pidof ssh
if [[ $? -ne 0 ]]; then
  echo Creating new tunnel connection
  createTunnel
fi

If I run any additional apps on the Pi, I'll need to:

• Create a tunnel (or modify the port range for the current one).
• Modify the Caddyfile on server and restart caddy with sudo systemctl restart caddy

Future explorations:

• Get Caddy or perhaps TCPProxy to access other services on Pi, not just webapps
• Map an entire port range so that the above two steps are not needed everytime I add a new Webapp on the Pi
• Test for IPv6 handling throughtout the stack

Talk to me on Twitter for issues/suggestions.

Update - OpenSSH also comes with tun/tap devices:
https://wiki.archlinux.org/index.php/VPN_over_SSH#Enable_forwarding_for_the_TUN_device
and there is also PPP:
https://wiki.archlinux.org/index.php/VPN_over_SSH#Using_PPP_over_SSH

But this is for OpenSSH. Dropbear SSH didn't support this.

I saw you were reaching out for suggestions, a simpler and more self-contained approach is to use inlets which has built-in TLS and TCP support for tunnelling services. Quick-start: Tunnel a private SSH server over inlets PRO

Not free, but fairly low price for personal use, and can be used with K8s also.

Hello @alexellis, I'm a fan of your work. A quick question though: Does inlets Pro support dead connection detection at the client side? Which is basically the underlying cause of this issue on inlets (Non Pro).

In fact, I have an inlet (Not Pro) instance started 1 month ago and currently running dead in the other room. It works for the first 10 minutes or so, then it hangs, because the websocket transport connection is gone.

I hope there is a way for inlet (and Pro) client&server to automatically recover from dead connections. Is that possible?

Consider using autossh to automatically restart SSH sessions and tunnels in case of failure.
I use it for many years. It is available via apt.

You can probably get the same result with less work by using socat, which is a fantastic tool. Basically just one line to achieve this – not sure about the exact syntax right now, but something like this:

socat tcp4-listen:1234 tcp4-connect:domain.com:22

@nirui - nobody has reported that, other than the user in China, where I suspect the government is interfering with packets.

Feel free to try inlets PRO, get a free trial and see how it goes for you.

You can also generate a systemd unit file for your local computer / gateway.

+1 for autossh. It also had it quirks to set up, specifically it sometimes needed to be restarted and I had to configure its polling interval. Reasons lost in the mists of time. See below my systemd .service file. Also on a pi, as it happens.

[Unit]
Description=Autossh
After=local-fs.target network.target

[Service]
Environment="AUTOSSH_POLL=60"
ExecStart=/usr/bin/autossh -R 11080:localhost:80 -N [email protected]
WorkingDirectory=/tmp/
User=pi
Restart=always
RestartSec=3s

[Install]
WantedBy=multi-user.target

Hi, instead of the bash script in crontab you might want to create a systemd unit to manage the reverse ssh proxy connection.

In your case something like this would probably work:

[Unit]
Description=SSH Reverse proxy
After=network.target
StartLimitIntervalSec=0

[Service]
Type=simple
Restart=always
RestartSec=1
User=root
WorkingDirectory=/
ExecStart=/usr/bin/ssh -N -T -R 10080:localhost:80 [email protected]
ExecStop=/bin/kill -9 $MAINPID

[Install]
WantedBy=multi-user.target

Assuming that you install that file as /etc/systemd/system/ssh-reverse-proxy.service and run systemctl daemon-reload, the advantage is that you can now enable it on boot by running systemctl enable ssh-reverse-proxy and it will be also restarted in case it goes down. You can now start and stop it via systemctl (start|stop) ssh-reverse-proxy and can read its logs via journalctl (es: journalctl -fu ssh-reverse-proxy to stream its logs).

:)

EDIT: apparently @avisagie beat me on time :)

Why not use Tailscale instead if your goal is to access internal services?

btw, WireGuard would also be a excellent ssh replacement for this case 👍

Sure u can do so, just perpetually add you home server access latency to the server response time.

I created https://github.com/fasmide/remotemoe for this exact use-case :)

Nebula/WireGuard VPN + nginx works well.

Nice stuff... but once you had the Hetzner VPC, you could have dropped ZeroTier on it, dropped another ZeroTier instance on your Pi, plumbed them both onto a common private network and then just used Caddy to reverse proxy the web server (or any other service) directly from your home over the private ZeroTier mediated network. All you need at home is port 9993 open (I think) for ZeroTier to work. It provides an encrypted & authenticated SDN-like connection between your systems. And frankly, it's a lot easier than IPsec or the other VPN style solutions and at least as strong if not stronger.

If you just want private access to the Pi based resources you can put ZeroTier on your phone, tablet or laptop and just access things directly that way, no need for the VPC.

As I say, nice work though. Hadn't seen caddy before. Must have a closer look.

btw, WireGuard would also be a excellent ssh replacement for this case

I use wireguard with nginx reverse-proxy to accomplish the same thing. Now I'm curious as to which is faster.

It's awesome to see SSH being used for more than just logging in to servers.
A few years back i wrote a custom ssh server to do this exact thing, altho my use case was webhook development rather than connecting a pi. other peeps have found it useful and it's turned into a bit of a side project so if anyone wants to try a reverse ssh tunnel on it before putting in the work to setup their own server, run ssh -R 80:localhost:8080 ssh.localhost.run and you'll have a reverse tunnel wired up to localhost:8080 with an internet accessible domain name returned in your terminal.

+1 @avisagie

@nileshtrivedi I wonder is this technique can be used to actually enable me to ssh into my rpi using a subdomain, similar to how you are serving html via port 80.

I try to ssh [email protected] but it didn't work...