Skip to content

Instantly share code, notes, and snippets.

@amy-langley
Created August 14, 2019 16:38
Show Gist options
  • Save amy-langley/0dffa939875cdc22ce3d7952aff17064 to your computer and use it in GitHub Desktop.
Save amy-langley/0dffa939875cdc22ce3d7952aff17064 to your computer and use it in GitHub Desktop.
like move_internal.rb, but does not loop, just finds the first few and quits immediately. it needs to be fast so that it moves things before reduce workers can pick them up.
# this assumes the rails console has preloaded the resources; accordingly,
# you should invoke this script as
#
# rails c < move_small.rb
begin
args_list = []
jobs_list = []
puts 'finding jobs'
Sidekiq::Queue.new('internal').each do |job|
if (job.klass=='ReduceWorker') && (job.args[0]==11235)
args_list.push(job.args)
jobs_list.push(job)
if(args_list.size % 20 == 0)
print '.'
# .flush
end
end
if (args_list.size == 100)
puts 'deleting jobs'
ct = 0
jobs_list.each do |job|
ct = ct + 1
print '.' if (ct % 20 == 0)
job.delete
end
jobs_list = []
puts "\npushing 100 jobs"
Sidekiq::Client.push_bulk('queue' => 'holding', 'class' => ReduceWorker, 'args' => args_list)
args_list = []
puts "\nfinding jobs"
break
end
end
ensure
puts 'cleaning up'
puts 'pushing remaining jobs'
Sidekiq::Client.push_bulk('queue' => 'holding', 'class' => ReduceWorker, 'args' => args_list)
puts 'deleting jobs'
ct = 0
jobs_list.each do |job|
ct = ct + 1
print '.' if (ct % 20 == 0)
job.delete
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment