Capturing my steps for creating CIDR network lists (ipset) suitable for use with firewalls such as iptables and nftables.

For example, I reference this approach in my Secure defaults for Debian sshd_config and MFA gist.

The ipsets that I build in this gist reduce the permitted IPv4 hosts from ~4.28 billion to ~47 million, a reduction of ~98.88%, which helps to reduce the attack surface but still lets in some traffic including your preferred IPS/networks.
This approach is not as good as, or a replacement for, setting up a VPN or Bastion setup, or implementing Zero Trust Network Access (ZTNA), or reducing the host list to only trusted hosts/networks. These topics can be tricky for dynamic IP addressing setups and will be the subject of a future Gist.

Step 1 - determine relevant ASN(s)

Glossary: AS refers to the network itself, and ASN refers to the number that identifies that network in the context of Internet routing. Autonomous System and Autonomous System Number respectively.

You need to choose ASN's that are relevant to your specific situation.
My logic is to look at which popular ISP's / networks that I have easy access to in my local region. In my case German Telekom, German Vodafone and Telefónica Germany.

For certain hosts/services, I want to be able to connect from those networks, and drop traffic outside those networks.

From these ISP's / networks, I check for an example assigned public IP address and then check the ASN info.
For example: https://ipinfo.io/176.3.20.40 which is Telefónica Germany AS6805

Step 2 - obtain AS data

For my use case:
Deutsche Telekom: https://search.dnslytics.com/bgp/as3320
Vodafone / Unity Media: https://search.dnslytics.com/bgp/as3209
Telefónica Germany GmbH & Co. OHG: https://search.dnslytics.com/bgp/as6805 which includes mobile provider blau.de

Open browser tab
Browse to https://search.dnslytics.com/
Open the browser dev tools and select the Network tab
Make the request for a given ASN
Filter the results to "a.dnslytics"
You should see a json request to a.dnslytics.com/v1/report/asn, check the payload
Later we can either copy the raw response to the clipboard OR
save to file e.g. dnslytics-AS3209.json

Pasting long/large data into the terminal

💡 Skip this section if you prefer to save the json content to file and work with the files directly.

I wanted to avoid saving/downloading the json files and then uploading to a remote host, so I went with the approach of copy & pasting the local browser raw json response data into a remote terminal.

I noticed that I hit a limit when pasting the raw json into a remote terminal. Related: Is there any limit on line length when pasting to a terminal in Linux?

I crafted following shell snippet workaround. I tested the invocation from sh, dash, bash and zsh*.

First, set a file target file name: asn_file=dnslytics-AS3320.json

Second: Copy & paste this snippet into your terminal and press ENTER if necessary to invoke it:

( #<subshell>
saved=$(stty -g) && echo 'term settings saved.'
# trap CTRL+C/INT so we can reset the terminal settings
trap 'printf "\ncat done\n"; stty "$saved" && echo "term settings restored."' INT
# Disable canonical mode (raw mode) and enable CR to LF translation
stty -icanon icrnl
echo "cat is waiting for your paste, pasted data is saved to: $asn_file"
echo 'after pasting, press ENTER for an LF and then CTRL+C to finish.'
cat > "$asn_file"
) #</subshell>

💡 zsh* requires setopt interactive_comments otherwise it throws: zsh: parse error near `\n'

Observations on the shell snippet

The shell snippet is wrapped in a subshell, which groups the commands together and keeps the trap and other scope aspects local to the subshell. In the bash invocation, the subshell also mitigates the stty commands from changing the parent shell. This mitigation doesn't apply for the sh invocation hence the trap to reset the terminal and increase the portability of the snippet.

Step 3 - filter AS data

Using your preferred shell, utilise jq and iprange commands

AS3209 Vodafone / Unity Media

$ < dnslytics-AS3209.json jq -r '
  .asn.prefixesv4[] | 
  select(.shortname | test("vodafone"; "i") or test("unity"; "i")) |
  select(.country == "DE") |
  .prefix
' | iprange | sort -h > AS3209-vodafone-unity-filtered-optimized-sorted.ipset

This ipset would permit 8,956,018 hosts.

AS3320 Deutsche Telekom

$ < dnslytics-AS3320.json jq -r '
  .asn.prefixesv4[] |
  select(.shortname | contains("Telekom")) |
  select(.country == "DE") |
  .prefix
' | iprange | sort -h > AS3320-telekom-filtered-optimized-sorted.ipset

This ipset would permit 33,702,338 hosts.

AS6805 Telefónica Germany

$ < dnslytics-AS6805.json jq -r '
  .asn.prefixesv4[] |
  select(.shortname | test("telefonica"; "i") and (test("o2"; "i") | not)) |
  select(.country == "DE") |
  .prefix
' | iprange | sort -h > AS6805-telefonica-filtered-optimized-sorted.ipset

This ipset would permit 5,188,560 hosts.

Observations on the `jq` logic

For my use case, I only want country == "DE" (Germany) and to filter out various shortname patterns which are uninteresting. For example the ISP's/networks host networks/infrastructure for their customers and those CIDR's can be filtered out.

For my use case, it is important to include the dynamic and static ranges for the AS/ISP, so if I'm on those networks, I'll be able to connect to the relevant services.

I calculated the permitted host totals with the following shell snippet:

for network in $(<AS6805-telefonica-filtered-optimized-sorted.ipset); do ipcalc $network ; done | awk '/^Hosts\/Net:/ {total += $2} END {print total}'
5188560

Step 4 - verify the ipset files contain relevant IP's

This step verifies that IP address used in step 1 are still inside the ipset files (CIDR's) that you've created.
The grepcidr command is utilised.

For example using the IP address example from step 1:

grepcidr -f AS6805-telefonica-filtered-sorted-iprange.ipset <(echo 176.3.20.40) 1>/dev/null && echo 'IP address was found inside the specified CIDRs' || echo 'NO MATCH'

Reference: Check if IP Belongs to a CIDR

Step 5 - using ipset files in a firewall

`iptables`

Consider how you are handling and persisting your firewall config
I'm currently using ipset-persistent and iptables-persistent plugins on Debian bookworm
sudo aptitude install ipset-persistent iptables-persistent
Create a network based ipset (note the hash:net)
sudo ipset create allowed_ssh_networks hash:net
Load the networks into the ipset
while read -r network; do echo "adding $network"; sudo ipset add allowed_ssh_networks "$network"; done < ~/ipset/merged.ipset
In my setup, I INSERTED a new rule to allow localnet ssh iptables -I INPUT 9 -p tcp -s 192.168.170.0/24 -m tcp --dport 22345 -j ACCEPT
I removed the previous allow ssh rule
sudo iptables -D INPUT 10
I tested that only localnet could connect to ssh on port 22345
Then I added the ipset rule
sudo iptables -A INPUT -p tcp --dport 22345 -m set --match-set allowed_ssh_networks src -j ACCEPT
Then I verified that I could connect from networks in the allowed_ssh_networks ipset
Then I connected to a VPN and verified I could NOT connect from networks outside the allowed_ssh_networks ipset
Save the ipset and iptables state
systemctl stop fail2ban.service && netfilter-persistent save && systemctl start fail2ban.service
Reboot and re-test everything is as expected

TODO Make some notes for nftables
TODO Make some notes for Proxmox

Considerations for automation

TODO

kyle0r/Creating an ipset from AS json.md