Skip to content

Instantly share code, notes, and snippets.

@twkm

twkm/robots-nofollow-header.md

Created July 5, 2016 14:38
Show Gist options
  • Save twkm/3210f09c5587876b9954d038f46b157d to your computer and use it in GitHub Desktop.
Save twkm/3210f09c5587876b9954d038f46b157d to your computer and use it in GitHub Desktop.
Download ZIP
Raw
robots-nofollow-header.md

Conditional X-Robots-Tag Header

This apache configuration can be added to the .htaccess file of a site to automatically keep staging and development versions from being indexed by Google and other search engines. No more worrying about whether or not noindex is keeping live sites from being crawled by search engines!

.htaccess configuration

# Check HTTP_HOST variable and set noindex, nofollow for hosts other than the live site
<IfModule mod_rewrite.c>
	RewriteCond %{HTTP_HOST} ^domain\.com$
	RewriteRule ^ - [E=live_env:true]
</IfModule>
<Files ~ ".*">
	Header set X-Robots-Tag "noindex, nofollow" env=!live_env
</Files>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment