Skip to content

Instantly share code, notes, and snippets.

@perplexes
Created June 4, 2012 16:00
Show Gist options
  • Save perplexes/2869222 to your computer and use it in GitHub Desktop.
Save perplexes/2869222 to your computer and use it in GitHub Desktop.
How to have multithreaded workers in delayed_job 3.0.3 and Ruby 1.9.3 and Rails 3.2.2
task :workers => :environment do
module Delayed
class Worker
def name_with_thread_id(*a, &b)
name_without_thread_id(*a, &b) + " thread:#{Thread.current.object_id}"
end
alias_method_chain :name, :thread_id
end
end
Rails.logger.info "Running threaded worker env."
Delayed::Worker.lifecycle.around(:execute) do |*args, &block|
thread_num = (ActiveRecord::Base.connection_config[:pool] || 1)
threads = []
Rails.logger.info "Using #{thread_num} threads for the worker."
thread_num.times do
threads << Thread.new(&block)
end
threads.each(&:join)
Rails.logger.info "End of threads."
end
Rake::Task["jobs:work"].execute
end
@ayqazi
Copy link

ayqazi commented Jan 15, 2014

Mostly seems to work on Ruby 1.9.3, Rails 3.2.16 and delayed_job 4.0.0 too.
Issues are:

It won't invoke the rake task - fixed by simply including the rest of the code inside an initializer (before the delayed_job config)

Using the ActiveRecord connection pool doesn't work since giving each connection in the pool to a thread leaves none for the master process - fixed by using an explicit value for thread_num or by negating 1 from thread_num.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment