Skip to content

Instantly share code, notes, and snippets.

@hpohlmeyer
Last active July 16, 2017 13:08
Show Gist options
  • Save hpohlmeyer/bee5a36e430becb9aacfc5815be6c8b6 to your computer and use it in GitHub Desktop.
Save hpohlmeyer/bee5a36e430becb9aacfc5815be6c8b6 to your computer and use it in GitHub Desktop.
Prevent a domain from being indexed and listed on by search engines

Hide domain from search engines

Preventing the url from being indexed through the robots.txt is not enough. It may still be listed by search engines, unless you add the right header to you file, in the .htaccess.

Both files have to be put into the root directory of your domain.


# .htaccess
Header set X-Robots-Tag "noindex, nofollow"

# robots.txt
User-agent: *
Disallow: /


Sources:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment