Harden Portainer and Apache2 Reverse Proxy with fail2ban

apache2_portainer_fail2ban.md

Fail2ban and Portainer with Apache2 Reverse Proxy

Prerequsits

• Ubuntu 22.04
• Portainer with Remote access
• apache2 as reverse proxy e.g. as described here
• fail2ban and e.g. iptables are installed
• Portainer is accesible via https://YourDomain/portainer/

User --> https --> Apache2 --> http(s) --> Portainer

IMPORTANT NOTE

Usually it is a bad idea to make portainer accessible via internet as front end. It has a HUGE SECURITY RISK, please know what you are doing!

Short how-to harden your Portainer Server with Fail2Ban

Install fail2ban:

sudo apt update && sudo apt install fail2ban -y

Create the Portainer-filter:

sudo nano /etc/fail2ban/filter.d/apache-portainer.conf

Portainer will not write Host IP in authentication errors in logs, but you have chance to track it via apache2 access.log as 401 and 422 errors.

Paste the following lines in /etc/fail2ban/filter.d/apache-portainer.conf, this will cover GUI Failed login attempts:

[Definition]
failregex = ^<HOST>.+?\/portainer\/api\/.+? HTTP\/\d+(?:\.\d+)?\" 4(?:01|22)
ignoreregex = 
    
[Init]
datepattern = \[%%d/%%b/%%Y:%%H:%%M:%%S %%z\]

Create a new jail:

sudo nano /etc/fail2ban/jail.d/apache-portainer.local

Paste the following rows:

[apache_portainer]
backend = auto
enabled = true
port = 80,443
protocol = tcp
filter = apache_portainer
# Number of retrys before to ban. Portainer produces from 2 to 5 log entries per request or failed login.
maxretry = 10
#time in seconds
bantime = 36000
findtime = 36000
# Log path, on Ubuntu usually is following
logpath = /var/log/apache2/access.log

Re-start the fail2ban-service:

sudo service fail2ban restart

and enjoy your Portainer!

I have a fail2ban filter like this:

[INCLUDES]
before = common.conf

[Definition]
failregex = ^<HOST> - - .*\/portainer\/api.*HTTP/[0-9]+(.[0-9]+)?" 401
            ^<HOST> - - .*\/portainer\/api\/auth.*HTTP/[0-9]+(.[0-9]+)?" 422
     
ignoreregex =

[Init]
datepattern = \[%%d/%%b/%%Y:%%H:%%M:%%S %%z\]

in my log I have something like this:

176.20.18.39 - - [30/Oct/2023:15:22:47 +0100] "POST /portainer/api/auth HTTP/1.1" 422 59 "https://myhost.myh.org/portainer/" "Mozilla/5.0 (Linux; Android 13; 2201122G Build/TKQ1.220807.001; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/118.0.0.0 Mobile Safari/537.36 Home Assistant/2023.10.2-11484 (Android 13; 2201122G)" "-"

but fail2ban does not trigger it's rules... any idea why?

thanks!!!

I have tried various regex but I can't find the error

Ok, you can test date pattern by command:

fail2ban-regex -d '\[%%d/%%b/%%Y:%%H:%%M:%%S %%z\]' /var/log/apache2/access.log "<HOST>"
...
Date template hits:

Lines: 4533 lines, 0 ignored, 4533 matched, 0 missed

and it seems works for date.
The whole line would be:

fail2ban-regex -d '\[%%d/%%b/%%Y:%%H:%%M:%%S %%z\]' /var/log/apache2/access.log '^<HOST> - - .*\/portainer\/api\/auth.*HTTP/[0-9]+(.[0-9]+)?" 422'
...
Date template hits:

Lines: 4621 lines, 0 ignored, 2 matched, 4619 missed
# Catched my 2 login attempts

Now if I test with config in /etc/fail2ban/filter.d/apache-portainer.local:

[Definition]
failregex = ^<HOST> - - .*\/portainer\/api.*HTTP/[0-9]+(.[0-9]+)?" 401
            ^<HOST> - - .*\/portainer\/api\/auth.*HTTP/[0-9]+(.[0-9]+)?" 422
ignoreregex = 

[Init]
datepattern = \[%%d/%%b/%%Y:%%H:%%M:%%S %%z\]

it will catch my login attempts

fail2ban-regex /var/log/apache2/access.log /etc/fail2ban/filter.d/apache-portainer.local -v --print-all-matched

Running tests
=============

Use   failregex filter file : apache-portainer, basedir: /etc/fail2ban
Use      datepattern : \[%d/%b/%Y:%H:%M:%S %z\] : \[Day/MON/Year:24hour:Minute:Second Zone offset\]
Use         log file : /var/log/apache2/access.log
Use         encoding : UTF-8


Results
=======

Failregex: 3 total
|-  #) [# of hits] regular expression
|   1) [0] ^<HOST> - - .*\/portainer\/api.*HTTP/[0-9]+(.[0-9]+)?" 401
|   2) [3] ^<HOST> - - .*\/portainer\/api\/auth.*HTTP/[0-9]+(.[0-9]+)?" 422
|      192.168.0.87  Wed Nov 29 09:27:38 2023
|      192.168.0.87  Wed Nov 29 10:12:27 2023
|      192.168.0.87  Wed Nov 29 10:36:28 2023
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [4797] \[Day/MON/Year:24hour:Minute:Second Zone offset\]
`-

Lines: 4797 lines, 0 ignored, 3 matched, 4794 missed
[processed in 0.22 sec]

|- Matched line(s):
|  192.168.0.87 - - [29/Nov/2023:09:27:38 +0100] "POST /portainer/api/auth HTTP/2.0" 422 1147 "https://<DOMAIN>/portainer/" "Mozilla/5.0 (Android 13; Mobile; rv:120.0) Gecko/120.0 Firefox/120.0"
|  192.168.0.87 - - [29/Nov/2023:10:12:27 +0100] "POST /portainer/api/auth HTTP/2.0" 422 1147 "https://<DOMAIN>/portainer/" "Mozilla/5.0 (Android 13; Mobile; rv:120.0) Gecko/120.0 Firefox/120.0"
|  192.168.0.87 - - [29/Nov/2023:10:36:28 +0100] "POST /portainer/api/auth HTTP/2.0" 422 1147 "https://<DOMAIN>/portainer/" "Mozilla/5.0 (Android 13; Mobile; rv:120.0) Gecko/120.0 Firefox/120.0"
`-
Missed line(s): too many to print.  Use --print-all-missed to print all 4794 lines

I even update the Gist with a newer 1 line pattern. Try it out.

Hey this is great, but do you have any suggestion on how to get this to work with Portainer running on port 9443 with plain IP without a Reverse Proxy?

Not really, portainer itself do not log such kind of activities. But it will answer with correct http codes so you can catch it via reverse proxy.

Not really, portainer itself do not log such kind of activities. But it will answer with correct http codes so you can catch it via reverse proxy.

oh thanks for the response. Would that work with Nginx Proxy Manager? Or do you recommend instead to use OAuth with Authentik and implement 2FA?

Sure, this will work, you need to click on 3 dots and findout Proxy Host Number as on screenshot I have 15:

Then add corresponding log to your Fail2Ban, e.g. for this example it will be your_Path_to_NPM/data/logs/proxy-host-15_access.log.
You have to update fail2ban rule, as this tool has slightly different logs format, e.g. <HOST> will be somewhere in [Client <HOST>] area.