Skip to content

Instantly share code, notes, and snippets.

@jorgejaramillo
Created January 25, 2023 22:03
Show Gist options
  • Save jorgejaramillo/2122013d256212ccaec7780e0cf32d0a to your computer and use it in GitHub Desktop.
Save jorgejaramillo/2122013d256212ccaec7780e0cf32d0a to your computer and use it in GitHub Desktop.
User-agent: *
User-agent: Googlebot
User-agent: Googlebot-Mobile
User-agent: Googlebot-Image
User-agent: Googlebot-Video
User-agent: Googlebot-News
User-agent: AhrefsBot
User-agent: AhrefsSiteAudit bot
User-agent: Screaming Frog SEO Spider
Allow:
Disallow: /*/ctl/
Disallow: /admin/
Disallow: /App_Browsers/
Disallow: /App_Code/
Disallow: /App_Data/
Disallow: /App_GlobalResources/
Disallow: /bin/
Disallow: /Components/
Disallow: /Config/
Disallow: /contest/
Disallow: /controls/
Disallow: /Documentation/
Disallow: /HttpModules/
Disallow: /Install/
Disallow: /Providers/
Disallow: /Activity-Feed/userId/
Sitemap: https://www.viajesexito.com/sitemap.xml
Sitemap: https://www.viajesexito.com/Portals/1/index_destinos.xml
# End of robots.txt file
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment