Redis and Resque

What this guide will cover: the code you will need in order to include Redis and Resque in your Rails app, and the process of creating a background job with Resque.

What this guide will not cover: installing Ruby, Rails, or Redis.

Note: As of this writing I am still using Ruby 1.9.3p374, Rails 3.2.13, Redis 2.6.11, and Resque 1.24.1. I use SQLite in development and Postgres in production.

Background jobs are frustrating if you've never dealt with them before. Over the past few weeks I've had to incorporate Redis and Resque into my projects in various ways and every bit of progress I made was very painful. There are many 'gotchas' when it comes to background workers, and documentation tends to be outdated or scattered at best.

The first thing you need to know about setting up Redis and Resque is that things change depending on your environment. You can get away with very minimal setup for a development environment, but production environments such as Heroku require extra steps which can often be frustrating to find.

So let's get started with the steps to configure Redis and Resque in your Rails app.

Setup

Add the following gems to your Gemfile and bundle:

gem 'redis'
gem 'resque', require: 'resque/server'

Note: 'resque-scheduler' is an optional plugin for rescue which allows you to easily setup scheduled jobs. You don't need it if you just want to move basic processes into background jobs and queue them up when certain events happen in your code.

Documentation for these gems:

Adding these gems to our gem file will give us the tools we need to interact with Redis and Resque in in our code.

From here we only need to add a couple lines of code in a few different files. Starting off, we will need rake tasks to start our schedules and workers.

Create a lib/tasks/resque.rb file and add the following to it:

require 'resque/tasks'

namespace :resque do
  task :setup do
    require 'resque'
    ENV['QUEUE'] = '*'

    Resque.redis = 'localhost:6379' unless Rails.env == 'production'
  end
end

Resque.after_fork = Proc.new { ActiveRecord::Base.establish_connection } #this is necessary for production environments, otherwise your background jobs will start to fail when hit from many different connections.

desc "Alias for resque:work (To run workers on Heroku)"
task "jobs:work" => "resque:work"

The important bit is "Resque.redis = 'localhost:6379' unless Rails.env == 'production'". This line tells Rails that we're running Redis on port 6379 (which is the Redis default) unless were running our app in production. Great, but where do we tell it to look if we are running in production? That's next.

Create a config/initializers/redis.rb file and add this to it:

if Rails.env == 'production'
  uri = URI.parse(ENV["REDISTOGO_URL"])
  Resque.redis = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
end

Notice that we are using an environment variable to hold our Redis url for us. Also note this block of code is only going to be run when our environment is production. That's fine because we already told Rails where to look for Redis for every other environment.

And finally, we need to actually set the environment variable. We do this in config/environments/production.rb by adding the following line inside the AppName::Application.configure block like so:

YourAppNameHere::Application.configure do
  ENV["REDISTOGO_URL"] = 'redis://redistogo:[email protected]:1234/'

  #other stuff below
end

For the sake of clarity, the above Redis url is bogus. You'll need to replace it with a real one, which you can get for free by provisioning your Heroku app with Redis To Go Nano. When you visit your Redis To Go account, it should tell you what your url is.

Last but not least, we'll need a Procfile with some commands in it so we don't need to manually startup workers and schedulers on Heroku.

Create a Procfile at the base of your application (if you don't already have one) and add the following lines to it:

web: bundle exec rails server -p $PORT
worker: QUEUE=* bundle exec rake environment resque:work

And that's it for setup. Your app is now ready to create and schedule background jobs and workers in both production (assuming your using Heroku) and development.

Creating Background Jobs

Lets say we have a Rails app that sends an email out every time a user creates an account. If we had the code to send the email in our controller or model, the user would actually have to wait for Rails to send the email before he could continue after creating an account. This can be bad if the process of sending an email is slow (which it generally is).

Let's fix this by moving the code to send an email on account creation into a background job.

Create a app/jobs directory in your Rails app. This is where you will keep all your job classes. Inside the jobs directory, create an email_sender.rb file.

class EmailSender
  @queue = :emails_queue

  def self.perform(params)
    #code to send out emails
  end
end

Stick all the of the code you had to send the email in the self.perform method (you can pass in any parameters you need into this method, it's just a Ruby method after all.) Then, in place of where you used to keep the code for sending emails, put this:

Resque.enqueue(EmailSender, params[user_id: current_user.id])

Simple. Not only does it make your code look cleaner by moving responsibility out of controllers and models, but it also means that the code will execute in the background and the user will not have to wait for it to complete before he can continue.

Scheduled Jobs and Delayed Jobs

There is a gem called 'resque-scheduler' (https://github.com/bvandenbos/resque-scheduler) which allows for easily setting up delayed and scheduled jobs with Resque. It also has some decent documentation on getting it up and running and a couple example jobs to look at.

If you want to include it, add it to your gemfile and change your lib/tasks/resque.rake file to the following:

require 'resque/tasks'
require 'resque_scheduler/tasks'

namespace :resque do
  task :setup do
    require 'resque'
    require 'resque_scheduler'
    require 'resque/scheduler'

    ENV['QUEUE'] = '*'

    Resque.redis = 'localhost:6379' unless Rails.env == 'production'
    Resque.schedule = YAML.load_file(File.join(Rails.root, 'config/resque_scheduler.yml'))
  end
end

Resque.after_fork = Proc.new { ActiveRecord::Base.establish_connection }

desc "Alias for resque:work (To run workers on Heroku)"
task "jobs:work" => "resque:work"

Create a config/resque_scheduler.yml file and add all your scheduled jobs to it. Finally, add this line to your Procfile to include booting up scheduler automatically:

scheduler: bundle exec rake resque:scheduler

FIN

freddyb2/redis_and_resque.mkd

Redis and Resque