scofennell/Guest-Post-on-CSS-Tricks.markdown

Guest Post on CSS-Tricks

Forked from Chris Coyier's Pen Guest Post on CSS-Tricks.

December 5 2014

The Deal with WordPress Transients

Intro paragraph with subtle plug for Scott Fennell, WordPress theme and plugin developer from Anchorage, Alaska.

The web has gotten really "API-ish". By that I mean, almost every major site is pushing and pulling content to and from other sites. Consider:

A near-majority of ESPN.com seems to actually be Twitter.
Programmable Web is now tracking over 13,000 API's.
One of the biggest initiatives around WordPress for many months has been the WP-API.

In my own work, I've seen client and employer expectations grow to include the hope — nay, the assumption — that my skill set includes API integration. It's not enough to paste in an email signup form as in <iframe>. Rather, my agency needs them abstracted into a shortcode for easier maintenance. It's not sufficent to paste in a product slider via some 3rd party javascript, rather the designer expects precise control of the animations. Having a developer log in to a CDN to see if a domain is in development mode is too time-consuming, so we call the CDN and display the status in the dashboard.

A few years ago, Matt Mullenweg predicted that the future of WordPress was as a platform for apps. Full disclosure here: I love WordPress. I am a WordPress fanboy. That said, I truly think Matt had a point: Whatever the present or future of WordPress actually is, it provides with us with some phenomenal tools for API integrations. One of these tools is the Transients API. It's one of my favorite tools that ships with WordPress, and I'd like to share a few tricks with you. If you are new to transients in general, read on. If you already get the concept and want to skip to the nuts and bolts, feel free to do so.

One more thing: I'm going to assume you are familiar with making HTTP requests from a WordPress plugin, as it would be beyond the scope of this article to lay that out in detail. My examples will all use the WordPress HTTP API for making requests.

What do transients have to do with API integrations?

Any time you are calling a remote API, there's some extra latency, sometimes a lot. Transients allow you to cache the response that you get from the remote API, storing them nearby in your WordPress database (well, usually in the database; more on that later). Also, many API's have a rate-limit, meaning you are only allowed to make x amount of requests within a given time period. Transients allow you to request that API response from yourself, saving you many, many remote calls.

What are transients, exactly?

They're a way to cache information. Transients are a form of caching that takes place on the server, as opposed to browser caching. Think of a transient as an organism that has three components:

A key. A short string of text. The name of the organism.
A value. Any php variable. The body — the guts, if you will — of the organism.
A lifespan. Often expressed as a time constant such as `DAY_IN_SECONDS`. The amount of time for which we want this organism to live.

We can use transients to store the result of remote calls.

Huh… So what if the remote API changes its response? And won't all these transients clutter up my database?

That's the cool thing about transients: They expire automatically. If you attempt to retrieve a transient from your database after it has expired, WordPress will automatically delete it, preventing any clutter. At the same time, it will re-create it anew, ensuring that you have (reasonably) fresh content from the remote API.

Whoa, wait, WordPress will delete my valuable information?

Yeah, it will. Transients are absolutely not for storing data that can't be automatically re-created. Therefore, you wouldn't use a transient to store, for example, data that a user is entering in a form on your site.

Are transients just for remote calls?

Transients are for any chunk of information that takes a long time to generate. They will improve server latency for any routine more complex than retrieving a single database cell. That said, they are more code, and more code means more bugs. Therefore, I tend to reserve them for either remote calls or really large queries.

How does this work exactly?

Let's say you want a list of recent subscribers to your email newsletter. You use a third party service for email updates, and it exposes your list of subscribers via an API. Here's the sequence:

Right now, you'd have to call the API to ask it for the list of subscribers, since you don't have that information handy.
The API responds to your request with the list of subcsribers, and although it takes a few hundred milliseconds, you got the API response you needed.
You store that data on your server, in a transient, giving it a lifespan of `HOUR_IN_SECONDS`, meaning one hour.
For the next hour, every time a user loads your list of recent subscribers, it'll happen really quickly, since it's all stored in one big pile, in one cell in your database.
An hour goes by and a user tries to load your page. WordPress notices that the lifespan has expired, it deletes your local value, and you're back to step one of this process.

I still don't get it.

WROX book on plugin development

codex

Okay, I get it. Snippet me, bro.

Here's what transients look like in their most basic form:

<?php
/**
 * Get a list of email subscribers.
 *
 * @return object The HTTP response that comes as a result of a wp_remote_get().
 */
function css_t_subscribers() {

	// Do we have this information in our transients already?
	$transient = get_transient( 'css_t_subscribers' );
	
	// Yep!  Just return it and we're done.
	if( ! empty( $transient ) ) {
		
		// The function will return here every time after the first time it is run, until the transient expires.
		return $transient;

	// Nope!  We gotta make a call.
	} else {
	
		// We got this url from the documentation for the remote API.
		$url = 'https://api.example.com/v4/subscribers';
		
		// We are structuring these args based on the API docs as well.
		$args = array(
			'headers' => array(
				'token' => 'example_token'
			),
		);
		
		// Call the API.
		$out = wp_remote_get( $url, $args );
		
		// Save the API response so we don't have to call again until tomorrow.
		set_transient( 'css_t_subscribers', $out, DAY_IN_SECONDS );
		
		// Return the list of subscribers.  The function will return here the first time it is run, and then once again, each time the transient expires.
		return $out;
		
	}
	
}
?>

That's the basic routine: Check for the value locally, if you have it great, if not, grab it remotely and store it locally for next time.

But I was promised tricks.

Yes! I have some tricks to share now that you've seen the basics. I'm not going to bundle this up into a final example, because your treatment will likely need be tailored to your application. This is a grab bag, and I'm going to organize it around the three components of a transient that I explained earlier:

The name.
The content.
The lifespan.

Intriguing: These are also the three values that get passed to set_transient().

Tricks for naming your transient

This is by far the deepest part of my grab bag of tricks when it comes to transients. It's a little counter-intuitive, but naming your transients is the hardest thing about using them. The way you name your transients can open up a number of powerful opportunities, or break your plugin altogether.

Prefixing your transient names

It's helpful to be able to identify all of the transients that pertain to your plugin. The way to do this is to prefix them with your plugin namespace. This is also crucial for preventing collisions with other transients. That's why you see me doing css_t_subscribers instead of just subscribers in most of my examples here, where css_t is my imaginary prefix for css-tricks.com.

Changing the name in order to break the cache

There are two hard things in computer science: cache invalidation, naming things, and off-by-one errors.

Phil Karlton

Amazing! Out of the three hard problems in computer science, naming your transient involves two of them: Naming things, and cache invalidation. Naming your transient relates to cache invalidation because if you change the name of your transient, then WordPress won't be able to find it. This can be actually be a good thing, because it forces your plugin to call the remote API to get fresh data.

An occasion for this might be when you release a new version of the plugin. It stands to reason that if the plugin code has changed, then you would want your code to grab refreshed transients instead of continuing to serve the old ones that assume the old plugin code. One way to do this is to include the version number of your plugin in your transient name:

<?php

	// This is the version number for our plugin.
	CSS_T_VERSION = '3.1.4';

	function css_t_subscribers() {

		// Do we have this information in our transients already?
		$transient = get_transient( 'css_t_subscribers' . CSS_T_VERSION );
	
		[...]
		
		// Save the API response so we don't have to call again until tomorrow.
		set_transient( 'css_t_subscribers' . CSS_T_VERSION, $out, DAY_IN_SECONDS );
		
		[...]
		
	
	}
	
}
?>

I like to use a constant because constants are global, so I don't have to pass it into my function — it's already there. However, I make sure to prefix the constant itself, it in order to avoid collisions with other constants.

But then, out of nowhere, a pitfall! Since your transient name is changing each time you update your plugin, WordPress is never again going to have occasion to call your transients by their old names. This means they will never be deleted! Your transients will stay in your database forever, achieving the dreaded "clutter" status that people sometimes worry about when it comes to transients. That's one reason why it's important to prefix all of your transients. Given that prefix, you could automatically delete all of the transients that pertain to your plugin, perhaps even providing a helpful "purge cache" button in your plugin settings page. This is a bit of a delicate matter, so I'm going to save the details for later on in the article.

Let me come clean: I have never set up a test scenario to confirm that transients might someday build up as clutter in the manner I've described above, but it does seem to happen in some cases. Furthermore, a reading of the core get_transient() function would seem to support this hypothesis. WP Engine seems to agree as well.

Also worth noting: You could grab your plugin version dynamically, from the docblock at the top of your plugin, via get_plugin_data(). However, that script only loads in wp-admin. I would imagine that you could include it on the front end as well, although I haven't got around to trying that. I can't vouch for it.

Using magic constants for naming your transients

Php ships with some helpful variables called magic constants, the most useful of which are __CLASS__ and __FUNCTION__. These constants return a string for the name of the current php class, and the name of the current function, respectively. This can streamline your code when it comes to naming lots of things, including transients:

<?php

class CSS_T {

	function subscribers() {
		$transient_key = __CLASS__ .'_' . __FUNCTION__;
		[...]
	}

}

?>

You could easily combine this technique with the version number technique noted above.

Including your remote API key in your transient name

Most API's require you to create a unique API key, or some other way to associate your remote calls with your account. If you find that you need to change that API key, it stands to reason that your transients would want to point to remote calls using only the new key. For this reason, you might append your API key to your transient name:

<?php

function css_t_subscribers( $limit = 50 ) {

	$api_key = 'W$3Th&j8Ias76gF%^Fukg3%$Dy3ghd!@';

	$transient_name = __FUNCTION__ . '_' . $api_key';
	[...]
}

?>

You might be concerned about having your API key flying around cyberspace, as this could present a security concern. You could mitigate that by encrypting the API key, an approach that happens to have other benefits which I'll discuss shortly.

Including the remote url and request parameters in your transient name

A remote API is going to serve a different response based on the exact url you are querying. Some examples might be:

<?php
// Getting all subcribers VS getting just 50 of them.
$subscribers       = 'https://api.example.com/subscribers';
$fifty_subscribers = 'https://api.example.com/subscribers?limit=50';

// Getting all campaigns VS getting just the ones that have already sent.
$campaigns      = 'https://api.example.com/campaigns';
$sent_campaigns = 'https://api.example.com/campaigns?status=sent';
?>

It's very likely that your plugin will want to send many combinations of these parameters, so you'll likely expose them as function arguments. Given that, you can use those values to dynamically build unique transient keys for each query:

<?php
function css_t_subscribers( $limit = 50 ) {

	// The base url for getting subscribers.
	$url = 'https://api.example.com/subscribers';
	
	// Sanitize the limit variable.
	$limit = absint( $limit );
	
	// Add the limit variable to the url.
	$url = add_query_arg( array( 'limit', $limit ), $url );
	
	// Use the url in the transient name.
	$transient_name = __FUNCTION__ . '_' . $url';
	[...]
}
?>

CAUTION! There is a problem!

Man, we are like drunk with appending stuff to our transient keys here! It is absolutely reasonable to have all of these elements in your transient name:

Plugin namespace.
PHP class name.
PHP function name.
Plugin version number.
Remote API url.
Remote API request parameters.
Remote API key.

You could end up with a transient name that is well over 100 characters long, and that won't work. Why not? Because if you make your transient key longer than 40 characters, WordPress might not store the transient. This is because of a character limit in the options table in the WordPress database. It can be really, really easy to exceed this limit once you start prefixing, adding a version number, and adding some args. WordPress might increase this limit to 255 chars soon, but until then, the way to sidestep this issue is to compress your transient name via php's md5() function.

md5() can take virtually any string and compress it down to 32 characters — a new string that is gaurenteed to be unique to the string you fed it. The result is basically unreadable (it's a hash) but there's no reason you would need to read the transient key names, other than the prefix portion.

Given that we have as little as 40 characters to work with, md5() uses up 32 of them, that means we only have 8 left for our prefix. For the sake of code readability, I take a bow toward the third hard problem in computer science, off by one errors (see above), and give myself only 7 characters, just to be safe:

<?php
function css_t_subscribers( $limit = '50' ) {
	
	// The namespace for our plugin.  
	$namespace = css_t_namespace(); // Let's say this gives us the slug name, 'css_tricks';
	
	// Cut it down to a max of 7 chars.
	$namespace = substr($namespace, 0, 7 );

	// The base url for getting subscribers.
	$url = 'https://api.example.com/subscribers';
	
	// Sanitize the limit variable.
	$limit = absint( $limit );
	
	// Add the limit variable to the url.
	$url = add_query_arg( array( 'limit', $limit ), $url );

	// Build a transient name that is guarenteed to carry all the uniqueness we might want, and also be less than 40 chars.
	$transient_name = $namespace . md5( $url );

	[...]

}

?>

Who knew we could go on so long on the niche topic of naming transients. It's amazing how deep you can go with this, and it's all because the name can be changed in interesting ways so as to break the cache. But enough about names.

Tricks for storing data in a transient

Earlier in this article, I stated that transients have three components: A name, content, and a lifespan. It's time to look at the second portion, which is the content that you're caching in the transient.

It doesn't have to be a string

WordPress core tells us that we don't need to serialize our transient content before we store it. In other words, we aren't limited to storing simple values like strings or numbers. Rather, we can store entire arrays or objects, such as an HTTP response that comes as a result of wp_remote_request().

That said, just because you can store the entire response, that doesn't necessarily mean you should. It might help streamline your plugin if you parse the response a bit and only store the body, or even some subset of the body. Alternatively, maybe you have a good reason for storing the entire response, maybe because you want to react to the HTTP status code elsewhere in your plugin. It's up to you.

Not all data is worth storing

Speaking of HTTP status codes, one of the first things I'll do when making an API integration is read the documention and curate a list of HTTP status codes. In many API's, a status code in the 40x or 50x range means that I made a mistake in my plugin code, making a request that the API could not fulfill. There's probably no reason to store that in a transient, so I'll compare the response code to my list before saving:

<?php

// Get a list of subscribers from a remote API.
function css_t_subscribers() {
	
	[...]

	// Call the remote service.
	$response = wp_remote_get( $url );
	
	// Check our response to see if it's worth storing.
	if ( ! css_t_check_response( $response ) ) {
		return FALSE;
	}
	
	[...]

}

// Given an HTTP response, check it to see if it is worth storing.
function css_t_check_response( $response ) {

	// Is the response an array?
	if( ! is_array( $response ) ) { return FALSE; }
		
	// Is the response a wp error?
	if( is_wp_error( $response ) ) { return FALSE; }

	// Is the response weird?
	if( ! isset( $response['response'] ) ) { return FALSE; }
		
	// Is there a status code?
	if( ! isset( $response['response']['code'] ) ) { return FALSE; }
		
	// Is the status code bad?
	if( in_array( $response['response']['code'], css_t_bad_status_codes() ) ) { return FALSE; }

	// We made it!  Return the status code, just for posterity's sake.
	return $response['response']['code'];

}

// A list of HTTP statuses that suggest that we have data that is not worth storing.
function css_t_bad_status_codes() {
	return array( 404, 500 );
}

?>

Only storing the result of `GET` requests

I'm talking about RESTful API's here. In a restful API, you can make a request using different request types. Here are some of the most common:

GET - Used for getting data.
POST - Used for adding a row of data.
PUT - Used for editing an entire row of data.
PATCH - Used for editing part of a row of data.
DELETE - Used for deleting an entire row of data.

I keep talking about the wp_remote_request() family of functions, and guess what? They allow you to specify which type of request you're making. There is only one type of request whose response belongs in a transient, and that's a GET request. In fact, if you are making any other type of request, then you are likely trying to change data on the remote server, and that means that some of your transients might now be obsolete. This would be an occassion to dump all of the transients related to your plugin. I'll dig into how we might do that shortly.

In the example of our email API integration, every time someone signs up for my email list, that's my plugin sending a POST request to the remote API, to add them to my mailing list. I probably have a function in my plugin dedicated to calling that API. That function is going to detect what type of request I'm making and, if it's not a GET request, it's going to dump all my plugin transients.

This attitude assumes that data accuracy is more important than performance, and frankly that's not always going to be the case. Maybe you have a view that offers many hundreds of rows of data, and that data changes very frequently. In such a case, it would not be performant to be dumping your transients on every POST request made by your plugin.

Tricks when designating a lifespan for your transient

We're on to the third and final part of a transient: The lifespan. This can be expressed in a few different ways:

`set_transient( $name, $content, 3600 )` -- store the data for 3600 seconds, which is an hour.
`set_transient( $name, $content, 60 * 60 )` -- store the data for 60 minutes, which is an hour, only more readable.
`set_transient( $name, $content, HOUR_IN_SECONDS )` -- store the data for an hour, eminently readable. These ship with WordPress.

The "Mayfly" transient for debugging

A Mayfly is an insect that has an incredibly short lifespan. Consider the following transient:

set_transient( $name, $content, 1 )

That's a transient that will only last for one second! This transient is almost guarenteed to never be called from the database. It would have to be generated and then re-requested in less than a second. However, this introduces a helpful way to provide a sort of debug mode in your plugin. If you are trying to debug your code, one of the most common steps is to echo your variables to see if they reflect what you're expecting. This can be extremely frustrating with transients. You'd have to go into your API calls and comment out the transient logic in order to make sure you get fresh results for debugging, and then remember to un-comment them before deploying. Instead, I do this:

<?php

// If the user is a super admin and debug mode is on, only store transients for a second.
function css_t_transient_lifespan() {
	if( is_super_admin() && WP_DEBUG ) {
		return 1;
	} else {
		return DAY_IN_SECONDS;
	}
}

// Get subscribers, using a dynamic value for the transient time.
function css_t_subscribers() {

	[...]

	$lifespan = css_t_transient_lifespan();
	set_transient( $name, $content, $lifespan );

	[...]

}

?>

That said, if you have transients with a relatively long lifespan, such as DAY_IN_SECONDS, you're still going to get those old values until tomorrow. Not cool. That's what you need a way to easily purge all your plugin transients.

The purge

We need to select all of the transients that relate to our plugin, and then use the delete_transient() function to delete each one. In theory, we could delete them via SQL, but it's usually best to do things closer to the application level, and that rule definitely applies here. I'll explain why in a bit.

<?php

// Purge all the transients associated with our plugin.
function css_t_purge_transients() {

	global $wpdb;

	$prefix = css_t_get_transient_prefix();

	$t  = esc_sql( "_transient_timeout_$prefix%" );

	// Select the transients.
	$transients = $wpdb -> get_col( "SELECT option_name FROM { $wpdb -> options } WHERE option_name LIKE '$t'";

	// For each transient...
	foreach( $transients as $transient ) {

		// Strip away the WordPress prefix in order to arrive at the transient key.
		$key = str_replace( '_transient_timeout_', '', $transient );
		
		// Now that we have the key, use WordPress core to the delete the transient.
		delete_transient( $key );
    }

}
?>

You could call that function when a user clicks a button on your plugin settings page, when new posts are published, or perhaps whenever a widget is saved. It's up to you!

A word about object caching

Have you picked up on my cautious tone at different points in this article, eluding to the fact that transients are not always in the database? It's because of object caching.

Recently I was trying to debug an API integration for a client. I tried to use phpMyAdmin to inspect transient values in the database, only I couldn't find any. This is because the client was using object caching: That means their transients did not live in the database!

I'm not very smart on what exactly object caching is, but I do understand how it relates to transients:

Store your transients as normal, using `set_transient()`. If object caching is available, it will intercept transients on the way to the database and do something (presumably) better with them, though I don't know what.
Get your transients as normal, using `get_transient()`. If object caching is available, it will grab them from wherever it keeps them.
Delete your transients as normal, using `delete_transient()`. If object caching is available, it will delete them from wherever it keeps them.

Sorry for the vagueness here. If you want to know more, I'd suggest digging into WordPress core. For example, check out the source code for delete_transient(): You can see it checking for object caching before falling back to the normal WP options API.

Next steps

I wanted to keep this discussion focused on the Transients API, but the reality is that it is best used in conjunction with WordPress's HTTP API. If you are making a plugin that makes remote calls, you should use the WordPress HTTP API, you should abstract all those calls into one php class, and that class should use the WordPress Transients API before and after calling the remote service.

	// HTML in article should just be HTML, not escaped
	// CSS-Tricks doesn't do it exactly this way, but close enough.
	$("pre code").each(function() {
	var content = $(this).html();
	$(this).text(content);
	});

	@import "compass/css3";
	body {
	background: url(http://cdn.css-tricks.com/wp-content/themes/CSS-Tricks-10/images/bg.png);
	}

	article {
	background: white;
	color: #111;
	border: 20px solid #D6D5D5;
	padding: 40px;
	width: 80%;
	max-width: 700px;
	margin: 20px auto;
	}