Created
April 16, 2025 07:34
-
-
Save alanorth/a0d51d4253bd43a5598113c370c0d296 to your computer and use it in GitHub Desktop.
Generate list of bot networks for nginx mapping
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 12876 | (AS12876 - SCALEWAY S.A.S.) | |
|---|---|---|
| 132203 | (TENCENT-NET-AP-CN Tencent Building) | |
| 13238 | (YANDEX - YANDEX LLC) | |
| 136907 | (HWCLOUDS-AS-AP HUAWEI CLOUDS) | |
| 14061 | (DIGITALOCEAN-ASN) | |
| 14618 | (AMAZON-AES) | |
| 16276 | (OVH - OVH SAS) | |
| 16509 | (AMAZON-02) | |
| 203020 | (HOSTROYALE - HostRoyale Technologies Pvt Ltd) | |
| 204287 | (HOSTROYALE_TECHNOLOGIES - HostRoyale Technologies Pvt Ltd) | |
| 21859 | (ZEN-ECN) | |
| 23576 | (NHN-AS-KR NAVER Cloud Corp.) | |
| 24940 | (HETZNER-AS - Hetzner Online GmbH) | |
| 396982 | (GOOGLE-CLOUD-PLATFORM) | |
| 45102 | (ALIBABA-CN-NET Alibaba US Technology Co.) | |
| 37963 | (ALIBABA-CN-NET Hangzhou Alibaba Advertising Co.) | |
| 50245 | (SERVEREL-AS - Serverel Inc.) | |
| 55286 | (SERVER-MANIA) | |
| 6939 | (HURRICANE) | |
| 8075 | (MICROSOFT-CORP-MSN-AS-BLOCK) | |
| 150436 | (BYTEPLUS-AS-AP Byteplus Pte. Ltd.) | |
| 26548 | (PUREVOLTAGE-INC) | |
| 174 | (COGENT-174) | |
| 201341 | (CENTURION-INTERNET-SERVICES - trafficforce, UAB) | |
| 209709 | (CODE200-ISP1 - UAB code200) | |
| 132817 | (DZCRD-AS-AP DZCRD Networks Ltd) | |
| 207223 | (GLOBALCON - Global Connections Network LLC) | |
| 29802 | (HVC-AS) | |
| 396319 | (US-INTERNET-396319) | |
| 64286 | (LOGICWEB) | |
| 62874 | (WEB2OBJECTS) | |
| 212238 | (CDNEXT - Datacamp Limited) | |
| 46844 | (SHARKTECH) | |
| 197540 | (NETCUP-AS - netcup GmbH) | |
| 42366 | (TERRATRANSIT-AS - TerraTransit AG) | |
| 44546 | (ALFATELECOM-AS - ALFA TELECOM s.r.o.) | |
| 32934 | (FACEBOOK) | |
| 209366 | (SEMRUSH-AS - SEMrush CY LTD) | |
| 20473 | (AS-VULTR) | |
| 13335 | (CLOUDFLARENET) | |
| 133499 | (HOSTROYALETECHNOLOGIES-AS-AP HostRoyale Technologies Pvt Ltd) | |
| 134450 | (HOSTROYALETECHNOLOGIES-AS-AP HostRoyale Technologies Pvt Ltd) | |
| 207990 | (HR-CUSTOMER - HostRoyale Technologies Pvt Ltd) | |
| 39486 | (HOSTROYALE - HostRoyale Technologies Pvt Ltd) | |
| 44144 | (OMEGASOFT - HostRoyale Technologies Pvt Ltd) | |
| 133944 | (TRAFFICFORCE-INTERNET-SERVICES - trafficforce, UAB) | |
| 202496 | (PRIENAI-SCIENCES-INTERNET - trafficforce, UAB) | |
| 19084 | (COLOUP) | |
| 21769 | (AS-COLOAM) | |
| 17252 | (AS2-COLOAM) | |
| 205964 | (CODE200-180 - UAB code200) | |
| 203061 | (ITPROXIMUS - UAB code200) | |
| 205659 | (CODE200-ISP2 - UAB code200) | |
| 394474 | (WHITELABELCOLO393) | |
| 18779 | (EGIHOSTING) | |
| 31898 | (ORACLE-BMC-31898) | |
| 6142 | (SUN-JAVA) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/usr/bin/env bash | |
| OUTPUT_FILE=roles/dspace/files/nginx/datacenter-networks.conf | |
| # Write list of networks as comments | |
| cat datacenter-networks.tsv \ | |
| | xargs -P0 -I% echo "# %" \ | |
| > "$OUTPUT_FILE" | |
| PREFIXES_TEMP=$(mktemp) | |
| cut -f1 datacenter-networks.tsv \ | |
| | roles/common/files/scripts/asn2ip.py \ | |
| > "$PREFIXES_TEMP" | |
| echo "# Last update: $(date)" >> "$OUTPUT_FILE" | |
| roles/common/files/scripts/cidr-merge.py < "$PREFIXES_TEMP" \ | |
| | xargs -P0 -I% echo "% 'bot';" \ | |
| | sort >> "$OUTPUT_FILE" | |
| echo "Wrote $PREFIXES_TEMP" | |
| echo "Wrote $OUTPUT_FILE" |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment