Skip to content

Instantly share code, notes, and snippets.

@faizanayubi

faizanayubi/.htaccess

Created March 2, 2016 13:22
Show Gist options
  • Save faizanayubi/ef30c0846b07796c8696 to your computer and use it in GitHub Desktop.
Save faizanayubi/ef30c0846b07796c8696 to your computer and use it in GitHub Desktop.
Download ZIP
Block Bot
Raw
.htaccess
ErrorDocument 503 "Site temporarily disabled for crawling"
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^.*(bot|crawl|spider).*$ [NC]
RewriteCond %{REQUEST_URI} !^/robots\.txt$
RewriteRule .* - [R=503,L]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment