Created
June 4, 2012 16:00
-
-
Save perplexes/2869222 to your computer and use it in GitHub Desktop.
How to have multithreaded workers in delayed_job 3.0.3 and Ruby 1.9.3 and Rails 3.2.2
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
task :workers => :environment do | |
module Delayed | |
class Worker | |
def name_with_thread_id(*a, &b) | |
name_without_thread_id(*a, &b) + " thread:#{Thread.current.object_id}" | |
end | |
alias_method_chain :name, :thread_id | |
end | |
end | |
Rails.logger.info "Running threaded worker env." | |
Delayed::Worker.lifecycle.around(:execute) do |*args, &block| | |
thread_num = (ActiveRecord::Base.connection_config[:pool] || 1) | |
threads = [] | |
Rails.logger.info "Using #{thread_num} threads for the worker." | |
thread_num.times do | |
threads << Thread.new(&block) | |
end | |
threads.each(&:join) | |
Rails.logger.info "End of threads." | |
end | |
Rake::Task["jobs:work"].execute | |
end |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Mostly seems to work on Ruby 1.9.3, Rails 3.2.16 and delayed_job 4.0.0 too.
Issues are:
It won't invoke the rake task - fixed by simply including the rest of the code inside an initializer (before the delayed_job config)
Using the ActiveRecord connection pool doesn't work since giving each connection in the pool to a thread leaves none for the master process - fixed by using an explicit value for thread_num or by negating 1 from thread_num.