Last active
June 14, 2019 03:56
-
-
Save yuliyang/26adfb3be07a6d8fec80237d65775664 to your computer and use it in GitHub Desktop.
[WordPress] control how participating bots interact with the site using robots.txt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| User-agent: * | |
| Disallow: /wp-admin/ | |
| Disallow: /wp-includes/ | |
| Disallow: /wp-login.php | |
| # make sure to update theme folder | |
| Disallow: /wp-content/themes/theme-name | |
| Allow: /wp-admin/admin-ajax.php | |
| # to restraict the whole site | |
| User-agent: * | |
| Disallow: / | |
| # sitemap - this is redundant by tools like Google Search Console, choose one to use | |
| Sitemap: https://your.domain/sitemap_index.xml |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment