Skip to content

Instantly share code, notes, and snippets.

@amboutwe
Last active November 10, 2024 16:43
Show Gist options
  • Save amboutwe/0c71e42aa164238007d7ea88f174a93f to your computer and use it in GitHub Desktop.
Save amboutwe/0c71e42aa164238007d7ea88f174a93f to your computer and use it in GitHub Desktop.
Filters and example code for Yoast SEO robots or WP robots.txt
<?php
/********* DO NOT COPY THE PARTS ABOVE THIS LINE *********/
/*
* Replace Disallow with Allow Generated Robots.txt
* Credit: Unknown
* Last Tested: June 09 2020 using WordPress 5.4.1
*/
add_filter('robots_txt','custom_robots');
function custom_robots($output) {
$public = get_option( 'blog_public' );
if ( '0' != $public )
return str_replace('Disallow','Allow',$output);
}
<?php
/********* DO NOT COPY THE PARTS ABOVE THIS LINE *********/
/* Check which functions have been used to hook into `wp_robots`
* Credit: Yoast team
* Last Tested: Unknown
*/
\add_filter( 'wp_robots', 'list_hooks' , \PHP_INT_MAX);
function list_hooks( $robots ) {
global $wp_filter;
echo "<!-- This is a list of callback functions hooked into the 'wp_robots' filter:";
echo json_encode($wp_filter['wp_robots'], JSON_PRETTY_PRINT);
echo "-->";
return $robots;
}
/*
* Change meta robots to index all author archive pages
* Credit: Yoast development team
* Last Tested: Jul 20 2023 using Yoast SEO 20.11 on WordPress 6.2.2
*/
add_filter( 'wpseo_robots', 'yoast_seo_robots_change_author_archive' );
function yoast_seo_robots_change_author_archive( $robots ) {
if ( is_author() ) {
return 'index, follow, max-image-preview:large, max-snippet:-1, max-video-preview:-1';
} else {
return $robots;
}
}
<?php
/********* DO NOT COPY THE PARTS ABOVE THIS LINE *********/
/*
* Remove robots meta tags from Yoast SEO
* Credit: Yoast development team
* Last Tested: May 22 2024 using Yoast SEO 22.7 on WordPress 6.5.3
*/
add_filter( 'wpseo_robots', '__return_empty_array' ); // Removes Yoast ouptut and returns to WP core output
add_filter( 'wp_robots', '__return_false' ); // Disable core output. Use with above to disable completely.
// Older versions
add_filter( 'wpseo_robots', '__return_false' ); // Yoast SEO 19.8 or older
add_filter( 'wpseo_googlebot', '__return_false' ); // Yoast SEO 14.x - Deprecated
add_filter( 'wpseo_bingbot', '__return_false' ); // Yoast SEO 14.x - Deprecated
<?php
/********* DO NOT COPY THE PARTS ABOVE THIS LINE *********/
/*
* Change meta robots using Yoast SEO
* Credit: Yoast development team
* Last Tested: Nov 1 2021 using Yoast SEO 17.4 on WordPress 5.8.1
*/
add_filter( 'wpseo_robots', 'yoast_seo_robots_remove_search' );
function yoast_seo_robots_remove_search( $robots ) {
if ( is_search() ) {
return array();
} else {
return $robots;
}
}
<?php
/********* DO NOT COPY THE PARTS ABOVE THIS LINE *********/
/*
* Change meta robots using Yoast SEO
* Credit: Yoast development team
* Last Tested: Dec 12 2017 using Yoast SEO 9.2.1 on WordPress 5.0
*********
* DIFFERENT POST TYPES
* Post: Change 123456 to the post ID
* Page: Change is_single to is_page and 123456 to the page ID
* Custom Post Type: Change is_single to is_singular and 123456 to the 'post_type_slug'
Example: is_singular( 'cpt_slug' )
*********
* MULTIPLE ITEMS
* Multiple of the same type can use an array.
Example: is_single( array( 123456, 234567, 345678 ) )
* Multiple of different types can repeat the if statement
*********
* The return empty array removes the robots tag on the page
* Or you can return index/noindex follow/nofollow like
* return 'noindex, follow';
* Or
* return 'noindex, nofollow';
*/
add_filter( 'wpseo_robots', 'yoast_seo_robots_remove_single' );
function yoast_seo_robots_remove_single( $robots ) {
if ( is_single ( 123456 ) ) {
return array();
} else {
return $robots;
}
}
@ppcdias
Copy link

ppcdias commented Dec 2, 2022

@amarilindra Thank you!

@amboutwe
Copy link
Author

amboutwe commented Dec 7, 2022

@FatimaFurnitureSEO This is not the proper place to request support. Please check out our extensive help section or visit the free support forum. If you require further support, upgrading to our premium version provides you with access to our support team.

@ws256
Copy link

ws256 commented Dec 25, 2022

Hi all.
I also want to share my results.
You need to understand that WordPress itself changes the robot meta tag name='robots'.
First, disable all affected plugins and check.

To influence WordPress you need to use a filter wp_robots
Below is an example for a search results page:

<?php
add_filter('wp_robots', 'my_wp_robots_directives');
function my_wp_robots_directives( $robots ) {
  if ( is_search() ) {
	unset( $robots['max-image-preview'] );
	unset( $robots['noindex'] );
	$robots['index'] = true;
	return $robots;
	}
}
?>

But if you turn on Yoast SEO, it will fix this tag as it sees fit, according to Google's recommendations. Filter wpseo_robots

<?php
add_filter( 'wpseo_robots', 'yoast_seo_robots_remove_search' );
function yoast_seo_robots_remove_search( $robots ) {
  if ( is_search() ) {
    return null;
  }
}
?>

return false; does not work
return null; works
so the examples above might not work for you.

@L-X-T
Copy link

L-X-T commented Feb 15, 2023

Thank you a lot @amarilindra and @ws256.

To bad this function is not built in in Yoast. I totally agree @tomybyte, it's always very dangerous to update Yoast. I, for that reason, don't like the plugin, but it's still standard for SEO.

@HesterSchoeman
Copy link

HesterSchoeman commented Feb 8, 2024

Hi there, I need some help. We have Yoast installed on our website and I need to add a noarchive to the WP Fusion restricted posts. The condition is definately working . I tested the condition with an echo statement and it does add the ", noarchive" to the $robots string, but it does not update the robot. We can add the noarchive manually to the posts, but we need to automate it to ensure that it is present when there is a restriction on the post. What would the reason be for it not updating the robot?

add_filter('wpseo_robots', 'add_noarchive_directive');
function add_noarchive_directive($robots) {
 	global $post;
	$post_id = $post->ID;
    // Check if the current post is restricted for the user
     if (wp_fusion()->access->get_post_access_meta( $post_id )['lock_content'] ) {
			// Add 'noarchive' directive if the condition is met
        	$robots .= ', noarchive';
    }
    return $robots;
}

@LoganStrike
Copy link

Just a warning to anyone using these code snippets to remove the Yoast generated robots tag... before Yoast 19.8, the wpseo_robots filter removed the robots tag when the filter returned false, as shown in a few of the snippets above. But it appears the following PR changed that behaviour to add noindex, nofollow instead of removing the tag. Yoast/wordpress-seo#18488 This change is a major issue for SEO purposes on any site that thought they were removing the robots tag by returning false, as those sites/pages will be noindex,nofollow instead, as of Yoast 19.8. I can;t help but think this was an oversight on Yoast developers. I can't understand why they would change the behaviour of returning false from "don't show the robots tag" to "show a noindex,nofollow robots" tag.

@amboutwe
Copy link
Author

@LoganStrike Thanks for bringing this up again. I updated the code to remove the robots output. The first removes the Yoast changes and the second one removes the WordPress robots output. Use together to remove the robots meta tag completely.

Could you test the new code and let me know how it works in a real-world environment, preferably a non-production site?

@LoganStrike
Copy link

@amboutwe The updated code works in my tests. May I recommend also updating yoast_seo_robots_remove_search.php and yoast_seo_robots_remove_single.php?

@amboutwe
Copy link
Author

@LoganStrike Done!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment