Last active
May 22, 2019 17:51
-
-
Save virtualstaticvoid/8705533 to your computer and use it in GitHub Desktop.
A better `find_each` and `find_in_batches` for ActiveRecord, which preserves the original order of the query
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
module ActiveRelationExtensions | |
def find_each_with_order(options = {}) | |
find_in_batches_with_order(options) do |records| | |
records.each { |record| yield record } | |
end | |
end | |
# NOTE: any limit() on the query is overridden with the batch size | |
def find_in_batches_with_order(options = {}) | |
options.assert_valid_keys(:batch_size) | |
relation = self | |
start = 0 | |
batch_size = options.delete(:batch_size) || 1000 | |
relation = relation.limit(batch_size) | |
records = relation.offset(start).to_a | |
while records.any? | |
records_size = records.size | |
yield records | |
break if records_size < batch_size | |
# get the next batch | |
start += batch_size | |
records = relation.offset(start + 1).to_a | |
end | |
end | |
end | |
ActiveRecord::Relation.send(:include, ActiveRelationExtensions) | |
module ActiveRecord | |
module Querying | |
delegate :find_each_with_order, :find_in_batches_with_order, :to => :scoped | |
end | |
end |
This has a very serious bug that causes the method to drop 1 of them. It should just be offset(start) on line 30, not start + 1
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Thanks for this! Surprised to see that this isn't a part of Rails.