Caching and Pagination When implementing the cache system for [Gists.IO](http://github.com/sntran/gists.io), we hit a wall with pagination. Mostly due to our lack of knowledge on caching, and the difference between our definition and GitHub's one on pagination. So let this entry be the rubber duck.

Caching and Pagination.md

Lets talk about cache

Initially, we cache only individual gist using the key {:gist, gist_id, username} in which the first element of the tuple is the atom :gist to act as the tag, then the gist_id and the username of the owner of that gist.

When fetching for gists of a particular user, we perform a match on keys with pattern {:gist, :'$1', username} to retrieve all the cached gists by that username. All went well, or so we thought.

Lets talk about pagination

GitHub's API for fetching Gists will return a Link header when there are more results than the default per_page query, which is 30. The header has the following format when fetching gists for myself:

https://api.github.com/users/sntran/gists?page=3; rel=\"next\", 
https://api.github.com/users/sntran/gists?page=6; rel=\"last\",
https://api.github.com/users/sntran/gists?page=1; rel=\"first\",
https://api.github.com/users/sntran/gists?page=2; rel=\"prev\""

Gists.IO should also support page parameter. But our problem is that we would have a different per_page setting and we only show blog-like gists in the page. So even if GitHub returns 30 gists in page 1, there are not always 30 blog-like gists in the result. We still, however, want to show the full 30 blog-like gists in the page.