Skip to content

Instantly share code, notes, and snippets.

@thinkingcap
Created July 1, 2025 00:01
Show Gist options
  • Save thinkingcap/7252e4b201edcc832fab223ffc528436 to your computer and use it in GitHub Desktop.
Save thinkingcap/7252e4b201edcc832fab223ffc528436 to your computer and use it in GitHub Desktop.
nano /etc/lsws-rc/extra.d/mywebappname.rewrite.block-bot.conf
# Block unwanted bots by User-Agent
RewriteCond %{HTTP_USER_AGENT} "(scrapy|acunetix|BLEXBot|domaincrawler\.com|LinkpadBot|MJ12bot/v|majestic12\.co\.uk|AhrefsBot|TwengaBot|SemrushBot|Amazonbot|nikto|winhttp|Xenu\s+Link\s+Sleuth|Baiduspider|HTTrack|clshttp|harvest|extract|grab|miner|python-requests|ClaudeBot)" [NC]
RewriteRule .* - [F,L]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment