Created
September 24, 2010 02:49
-
-
Save rcanand/594782 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## /config/initializers/dynamic_job.rb | |
require 'heroku' | |
# base class for all jobs that you wish to automatically scale and go down in Heroku | |
class DynamicJob | |
#set a cap on maximum number of users ever - just in case. | |
MAX_CONCURRENT_WORKERS = 100 | |
def initialize | |
#assumption: the job class is new-ed up before enqueueing, which is always the case | |
workers_needed = [Delayed::Job.count + 1, MAX_CONCURRENT_WORKERS].min | |
if(ENV["RAILS_ENV"] == "production") | |
username = ENV['HEROKU_USERNAME'] | |
password = ENV['HEROKU_PASSWORD'] | |
app = ENV['HEROKU_APP'] | |
client = Heroku::Client.new(username, password) | |
client.set_workers(app, workers_needed) | |
end | |
end | |
# call this from the derived class when you are done. | |
def clean_up | |
#assumption: this method is always called from within a derived job class | |
workers_needed = [Delayed::Job.count - 1, 0].max | |
if(ENV["RAILS_ENV"] == "production") | |
username = ENV['HEROKU_USERNAME'] | |
password = ENV['HEROKU_PASSWORD'] | |
app = ENV['HEROKU_APP'] | |
client = Heroku::Client.new(username, password) | |
client.set_workers(app, workers_needed) | |
end | |
end | |
end | |
#example job class LoadImagesJob | |
class LoadImagesJob < DynamicJob | |
def initialize topic_id | |
@topic = Topic.find(topic_id) | |
#remember to call super() with parentheses to call superclass initializer without parameters | |
super() | |
end | |
def perform | |
@topic.load_images! | |
clean_up | |
end | |
end | |
#example enqueue | |
job = LoadImageJob.new topic.id # => calls some.do_something later | |
Delayed::Job.enqueue(job) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Yes, the value must be configured to be larger than anything you expect to see in production. The config is provided only for the off chance that a bug causes workers to never be decremented and we want to limit the number of workers so we dont lose our savings:)
Afaik, heroku will try to release workers gracefully, and will then send a force abort after some time. I dont remember the details, but I think an unfinished job whose worker is killed gets picked up by a free worker anyway, so it doesnt matter if an active worker is dropped. Lot of this is based on my testing a few weeks ago, so it may change.if heroku changes.