Skip to content

Instantly share code, notes, and snippets.

@detj

detj/robots.txt

Last active August 17, 2016 10:50
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save detj/cf3929251cf8fc8fa921ecf37feb4691 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save detj/cf3929251cf8fc8fa921ecf37feb4691 to your computer and use it in GitHub Desktop.
Download ZIP
robots.txt nginx config in one line
Raw
robots.txt
#Allow access to all User-agents:
location /robots.txt {return 200 "User-agent: *\nDisallow:\n";}
#Disallow access to every User-agent:
location /robots.txt {return 200 "User-agent: *\nDisallow: /\n";}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment