-
-
Save Xunnamius/6057a660d06bcf13cc1f478af9131423 to your computer and use it in GitHub Desktop.
# This version works with CF WAF (using zone rulesets) and obsoletes previous | |
# versions. This works will all CF account types. This action depends on curl | |
# and jp and will add/remove IPs from the $known_hostile_ips list. Creating the | |
# WAF rules need only be done once per zone. Creating the list need only be done | |
# once per account. | |
# | |
# Author: Bernard Dickens III (Xunnamius) | |
# | |
# Inspired by work from: Mike Rushton | |
# https://github.com/fail2ban/fail2ban/blob/master/config/action.d/cloudflare.conf | |
# | |
# 1. REQUIRES jp TO BE INSTALLED IN THE USER PATH! Install it here: | |
# https://github.com/jmespath/jp | |
# | |
# 2. ! IMPORTANT ! Set jail.local's permission to 640 because it contains one of | |
# your CF API tokens. Grab your account id, your api token, and your "hostile | |
# ip" list id before continuing (see end of file for details). | |
# | |
# 3. Create a new custom list. Name it known_hostile_ips. | |
# https://developers.cloudflare.com/waf/tools/lists | |
# | |
# 4. Ensure every zone you want fail2ban to protect has an enabled WAF ban rule | |
# referencing $known_hostile_ips. | |
# https://developers.cloudflare.com/waf/custom-rules | |
# | |
# 5. Use the fail2ban CLI and the Cloudflare dashboard/Traces to test and make | |
# sure everything is working properly. You may need to add a permission to | |
# your api token to ensure proper function. See the end of this file for | |
# details. | |
# | |
# To get your CloudFlare API Key: | |
# https://www.cloudflare.com/a/account/my-account | |
# | |
# CloudFlare API error codes: https://www.cloudflare.com/docs/host-api.html#s4.2 | |
# | |
# Note that if you're using Nginx, Apache, Litespeed, etc, you need to modify | |
# your logs and/or your filters such that the real client IP is being captured | |
# and not Cloudflare's IPs. | |
[Definition] | |
# Option: actionstart | |
# Notes.: command executed on demand at the first ban (or at the start of | |
# Fail2Ban if actionstart_on_demand is set to false). | |
# Values: CMD | |
# | |
actionstart = | |
# Option: actionstop | |
# Notes.: command executed at the stop of jail (or at the end of Fail2Ban) | |
# Values: CMD | |
# | |
actionstop = | |
# Option: actioncheck | |
# Notes.: command executed once before each actionban command | |
# Values: CMD | |
# | |
actioncheck = | |
# Option: actionban | |
# Notes.: command executed when banning an IP. Take care that the | |
# command is executed with Fail2Ban user rights. | |
# Tags: <ip> IP address | |
# <failures> number of failures | |
# <time> unix timestamp of the ban time | |
# Values: CMD | |
# | |
# API v4 WAF | |
actionban = curl -s -o /dev/null -X POST <_cf_api_prms> \ | |
-d '[{"ip":"<ip>","comment":"Created by fail2ban <name>"}]' \ | |
<_cf_api_url> | |
# Option: actionunban | |
# Notes.: command executed when unbanning an IP. Take care that the | |
# command is executed with Fail2Ban user rights. | |
# Tags: <ip> IP address | |
# <failures> number of failures | |
# <time> unix timestamp of the ban time | |
# Values: CMD | |
# | |
# API v4 WAF | |
actionunban = id=$(curl -s -X GET <_cf_api_prms> \ | |
"<_cf_api_url>?search=<ip>&per_page=1" \ | |
| { jp --unquoted 'result[0].id | not_null(@, `""`)' 2>/dev/null; }) | |
if [ -z "$id" ]; then echo "<name>: id for <ip> cannot be found"; exit 0; fi; | |
curl -s -o /dev/null -X DELETE <_cf_api_prms> \ | |
-d '{"items":[{"id":"'"$id"'"}]}' \ | |
<_cf_api_url> | |
_cf_api_url = https://api.cloudflare.com/client/v4/accounts/<cfaccountid>/rules/lists/<cfbanlistid>/items | |
_cf_api_prms = -H 'Authorization: bearer <cfapitoken>' -H 'Content-Type: application/json' | |
[Init] | |
# If you like to use this action with mailing whois lines, you could use the | |
# composite action action_cf_mwl predefined in jail.conf, just define in your | |
# jail: | |
# | |
# action = %(action_cf_mwl) | |
# # Your CF API Key | |
# cfapitoken = | |
# cfaccountid = | |
# cfbanlistid = | |
# Your Cloudflare User API Token. It will need "EDIT" level on the "Account | |
# Filter Lists" permission. | |
# https://dash.cloudflare.com/profile/api-tokens | |
cfapitoken = | |
# The identifier of the Cloudflare account used to update the hostile IP list. | |
cfaccountid = | |
# Your Cloudflare WAF "hostile ip" List id. You can find it using the API or or | |
# following the instructions here: | |
# https://community.cloudflare.com/t/what-token-permissions-for-ip-list-edits/525222/6 | |
# https://api.cloudflare.com/client/v4/accounts/:cfaccountid/rules/lists | |
# | |
# Note that even free CF accounts get 1 free list with 10,000 slots, yay! | |
# https://dash.cloudflare.com/:cfaccountid/configurations/lists | |
cfbanlistid = |
No idea... But this looks anyway artificial or rewritten somehow...
- there is no end for
cfip?family=inet6
(I don't see closing single quote at end) - there is no
cfip
(withoutfamily=inet6
)
Can you possibly try to rewrite actionban
to generate an error - curl
without -s -o /dev/null
(and possibly an intended error with ; exit 1
at end, to produce an output in fail2ban.log)?
'cloudflare-waf',
[ ['actionstart', ''],
['actionstop', ''],
['actioncheck', ''],
['actionban', 'curl -s -o /dev/null -X POST -H \'Authorization: bearer redacted\' -H \'Content-Type: application/json\' \\\n-d "[{\\"ip\\":\\"<cfip>\\",\\"comment\\":\\"Created by fail2ban mysqld-auth\\"}]" \\\nhttps://api.cloudflare.com/client/v4/accounts/redacted/rules/lists/redacted/items'],
['actionunban', 'id=$(curl -s -X GET -H \'Authorization: bearer redacted\' -H \'Content-Type: application/json\' \\\n"https://api.cloudflare.com/client/v4/accounts/redacted/rules/lists/redacted/items?search=<cfip>&per_page=1" \\\n| { jp --unquoted \'result[0].id | not_null(@, `""`)\' 2>/dev/null; })\nif [ -z "$id" ]; then echo "mysqld-auth: id for <cfip> cannot be found"; exit 0; fi;\ncurl -s -o /dev/null -X DELETE -H \'Authorization: bearer redacted\' -H \'Content-Type: application/json\' \\\n-d \'{"items":[{"id":"\'"$id"\'"}]}\' \\\nhttps://api.cloudflare.com/client/v4/accounts/redacted/rules/lists/redacted/items'],
['name', 'mysqld-auth'],
['actname', 'cloudflare-waf'],
['cfapitoken', 'redacted'],
['cfaccountid', 'redacted'],
['cfbanlistid', 'redacted'],
['cfip?family=inet6', 'fail2ban-python -c \'import sys; from fail2ban.server.ipdns import IPAddr; a = IPAddr(sys.argv[1]+"/"+sys.argv[2]); print(str(a))\' "<ip>" 64']]]
Sorry, before I knew to triple-tick I tried to tidy it and must have inadvertently removed some things I did not intend to remove!
Take an attentive look at fail2ban-python -c ...
.
It is not enclosed in ``.
Where in my example in https://gist.github.com/Xunnamius/6057a660d06bcf13cc1f478af9131423?permalink_comment_id=5047774#gistcomment-5047774 it is definitely in (the accent-chars are very important).
So either:
cfip = `fail2ban-python -c 'import sys; from fail2ban.server.ipdns import IPAddr; a = IPAddr(sys.argv[1]+"/"+sys.argv[2]); print(str(a))' "<ip>" 64`
Or:
cfip = $(fail2ban-python -c 'import sys; from fail2ban.server.ipdns import IPAddr; a = IPAddr(sys.argv[1]+"/"+sys.argv[2]); print(str(a))' "<ip>" 64)
`...`
or $(...)
is command substitution, whereas without it ...
is just a string.
@sebres, thank you!
I had tried it both ways (with the tick and without) and neither worked. Doing it with the $(...)
, however, has fixed it! The IPv6 address gets successfully added to the ban list. Unbanning it, however, does not remove it. And there are no errors in the F2B.log with the unban, either. 😄
Well, probably the URL https://api.cloudflare.com/client/v4/accounts/redacted/rules/lists/redacted/items?search=<cfip>&per_page=1
doesn't return the ID (URI may be incorrect due to unescaped /
in argument, however it is a bit unexpected).
Can you try this from command line? E.g. if it indeed doesn't work, check it with %2f64
instead of /64
.
@sebres having changed nothing, it's working now! Thanks so much for all of your help!
Following this thread. Is there a copy of the final draft solution for IPv6 that can be posted? I tried following the steps as written above and I'm still getting errors. My guess is I missed a step somewhere but I can't figure out where.
Here you go:
# This version works with CF WAF (using zone rulesets) and obsoletes previous
# versions. This works will all CF account types. This action depends on curl
# and jp and will add/remove IPs from the $known_hostile_ips list. Creating the
# WAF rules need only be done once per zone. Creating the list need only be done
# once per account.
#
# Author: Bernard Dickens III (Xunnamius)
#
# Inspired by work from: Mike Rushton
# https://github.com/fail2ban/fail2ban/blob/master/config/action.d/cloudflare.conf
#
# 1. REQUIRES jp TO BE INSTALLED IN THE USER PATH! Install it here:
# https://github.com/jmespath/jp
#
# 2. ! IMPORTANT ! Set jail.local's permission to 640 because it contains one of
# your CF API tokens. Grab your account id, your api token, and your "hostile
# ip" list id before continuing (see end of file for details).
#
# 3. Create a new custom list. Name it known_hostile_ips.
# https://developers.cloudflare.com/waf/tools/lists
#
# 4. Ensure every zone you want fail2ban to protect has an enabled WAF ban rule
# referencing $known_hostile_ips.
# https://developers.cloudflare.com/waf/custom-rules
#
# 5. Use the fail2ban CLI and the Cloudflare dashboard/Traces to test and make
# sure everything is working properly. You may need to add a permission to
# your api token to ensure proper function. See the end of this file for
# details.
#
# To get your CloudFlare API Key:
# https://www.cloudflare.com/a/account/my-account
#
# CloudFlare API error codes: https://www.cloudflare.com/docs/host-api.html#s4.2
#
# Note that if you're using Nginx, Apache, Litespeed, etc, you need to modify
# your logs and/or your filters such that the real client IP is being captured
# and not Cloudflare's IPs.
[Definition]
# Option: actionstart
# Notes.: command executed on demand at the first ban (or at the start of
# Fail2Ban if actionstart_on_demand is set to false).
# Values: CMD
#
actionstart =
# Option: actionstop
# Notes.: command executed at the stop of jail (or at the end of Fail2Ban)
# Values: CMD
#
actionstop =
# Option: actioncheck
# Notes.: command executed once before each actionban command
# Values: CMD
#
actioncheck =
# Option: actionban
# Notes.: command executed when banning an IP. Take care that the
# command is executed with Fail2Ban user rights.
# Tags: <ip> IP address
# <failures> number of failures
# <time> unix timestamp of the ban time
# Values: CMD
#
# API v4 WAF
actionban = curl -s -o /dev/null -X POST <_cf_api_prms> \
-d '[{"ip":"'"<cfip>"'","comment":"Created by fail2ban <name>"}]' \
<_cf_api_url>
# Option: actionunban
# Notes.: command executed when unbanning an IP. Take care that the
# command is executed with Fail2Ban user rights.
# Tags: <ip> IP address
# <failures> number of failures
# <time> unix timestamp of the ban time
# Values: CMD
#
# API v4 WAF
actionunban = id=$(curl -s -X GET <_cf_api_prms> \
"<_cf_api_url>?search=<cfip>&per_page=1" \
| { jp --unquoted 'result[0].id | not_null(@, `""`)' 2>/dev/null; })
if [ -z "$id" ]; then echo "<name>: id for <ip> cannot be found"; exit 0; fi;
curl -s -o /dev/null -X DELETE <_cf_api_prms> \
-d '{"items":[{"id":"'"$id"'"}]}' \
<_cf_api_url>
_cf_api_url = https://api.cloudflare.com/client/v4/accounts/<cfaccountid>/rules/lists/<cfbanlistid>/items
_cf_api_prms = -H 'Authorization: bearer <cfapitoken>' -H 'Content-Type: application/json'
[Init]
# If you like to use this action with mailing whois lines, you could use the
# composite action action_cf_mwl predefined in jail.conf, just define in your
# jail:
#
# action = %(action_cf_mwl)
# # Your CF API Key
# cfapitoken =
# cfaccountid =
# cfbanlistid =
# Your Cloudflare User API Token. It will need "EDIT" level on the "Account
# Filter Lists" permission.
# https://dash.cloudflare.com/profile/api-tokens
cfapitoken =
# The identifier of the Cloudflare account used to update the hostile IP list.
cfaccountid =
# Your Cloudflare WAF "hostile ip" List id. You can find it using the API or or
# following the instructions here:
# https://community.cloudflare.com/t/what-token-permissions-for-ip-list-edits/525222/6
# https://api.cloudflare.com/client/v4/accounts/:cfaccountid/rules/lists
#
# Note that even free CF accounts get 1 free list with 10,000 slots, yay!
# https://dash.cloudflare.com/:cfaccountid/configurations/lists
cfbanlistid =
cfip = <ip>
[Init?family=inet6]
cfip = $(fail2ban-python -c 'import sys; from fail2ban.server.ipdns import IPAddr; a = IPAddr(sys.argv[1]+"/"+sys.argv[2]); print(str(a))' "<ip>" 64)
@sebres thank you. I've tested this and it's still not working on my end but I will spend more time troubleshooting to see if I can yield the same results as @Staene. I did notice in the full solution that the end of your IPv6 cfip = statement there is an extra ` after the 64 that is not in the original statement above. I made that change on my end but it did not resolve the issue. Can you confirm if that ` should be there or not? Thanks again for all your help.
Fixed (the extra accent char must be removed), I just forgot to remove it by replacement `...`
with $(...)
Is anything working? If no, make sure your Cloudflare list is setup correctly and the API token has permission. You can check Manage Account > Audit Log to see if the API submissions are going through or Manage Account > Configuration > Lists to see if the list is populated.
If banning works but not unbanning, make sure jp is installed.
IPv4 works fine. As of now IPv6 is not.
I should add I run fail2ban in docker. I added jp to the crazymax/fail2ban image and after finally getting the IPv4 steps right I had success there. The biggest challenge was as you said, the API permissions - I missed the steps at the bottom of the instructions originally. I also run nginx proxy manager in docker and have specifically called out real_ip_header CF-Connecting-IP in the advanced tab of the proxy host i'm testing. Cloudflare is banning the appropriate IPv4 address as a result.
For IPv6 it shows the ban occur in the logs but the IP is never added to the cloudflare list. I'd love to post a log but I can't find any error.
Try changing [Definition?family=inet6]
to [Init?family=inet6]
I do not believe Definition
worked for me.
Right on! 👍
I've followed all the steps, but this isn't working for me, how can I troubleshoot maybe it's a token error
EDIT: confirmed it's not a token error, I called the API with the keys I used & got a 200
how can I troubleshoot
fail2ban.log contains the error message and few lines with info how exactly the actions command has been executed. This can help to identify the issue.
this is my file @ /etc/fail2ban/action.d/cloudflare-list.conf
[Definition]
actionban = curl -s -o /dev/null -X POST <_cf_api_prms> \
-d '[{"ip":"'"<cfip>"'","comment":"Created by fail2ban <name>"}]' \
<_cf_api_url>
actionunban = id=$(curl -s -X GET <_cf_api_prms> \
"<_cf_api_url>?search=<cfip>&per_page=1" \
| { jp --unquoted 'result[0].id | not_null(@, `""`)' 2>/dev/null; })
if [ -z "$id" ]; then echo "<name>: id for <ip> cannot be found"; exit 0; fi;
curl -s -o /dev/null -X DELETE <_cf_api_prms> \
-d '{"items":[{"id":"'"$id"'"}]}' \
<_cf_api_url>
_cf_api_url = https://api.cloudflare.com/client/v4/accounts/<cfaccountid>/rules/lists/<cfbanlistid>/items
_cf_api_prms = -H 'Authorization: bearer <cfapitoken>' -H 'Content-Type: application/json'
[Init]
cfip = <ip>
[Init?family=inet6]
cfip = $(fail2ban-python -c 'import sys; from fail2ban.server.ipdns import IPAddr; a = IPAddr(sys.argv[1]+"/"+sys.argv[2]); print(str(a))' "<ip>" 64)
Init params are in /etc/fail2ban/action.d/cloudflare-list.local
how can I troubleshoot
fail2ban.log contains the error message and few lines with info how exactly the actions command has been executed. This can help to identify the issue.
I'm not getting any errors related to the banning action
2024-09-05 15:13:01,875 fail2ban.actions [1856146]: NOTICE [sshd] Ban 10.10.10.10
You have to add this action to the jails action
parameter (in jail.local), like here:
https://github.com/fail2ban/fail2ban/blob/1ea8a6de58e6d982b5e42db5b1761281e91c51e1/config/jail.conf#L236-L237 (or below or above)...
To check the action is applied to the jail in config you can see the dump fail2ban-client --dp
...
And then restart (not just reload) the jail or fail2ban.
You have to add this action to the jails
action
parameter (in jail.local), like here: https://github.com/fail2ban/fail2ban/blob/1ea8a6de58e6d982b5e42db5b1761281e91c51e1/config/jail.conf#L236-L237 (or below or above)... To check the action is applied to the jail in config you can see the dumpfail2ban-client --dp
... And then restart (not just reload) the jail or fail2ban.
Thanks, this was what was missing from my jail.local
For those who are using a docker-compose. This is how I installed jp as the instructions require. Add this to your docker-compose.yaml file underneath the fail2ban container.
command: /bin/sh -c "apk add --update jp && fail2ban-server -f"
For this to work I have append > /dev/null to the end of the curl commands. For some reason my curl version was ignoring the -o param...
actionban = curl -s -o /dev/null -X POST <_cf_api_url> \
<_cf_api_prms> \
-d '[{"ip":"'"<cfip>"'","comment":"<cfcomment>"}]' \
> /dev/null
For a more confortable solution because I've already work with jq but not jp command I end using the solution in this thread https://community.cloudflare.com/t/does-cloudflares-fail2ban-action-still-work/640904/3 that display the { jq -r '.result[0].id' 2>/dev/null || tr -d '\n' | sed -nE 's/^.*"result"\s*:\s*\[\s*\{\s*"id"\s*:\s*"([^"]+)".*$/\1/p'; }
command to the pipe part of the actionunban.
Sorry! Fixed.