Created
September 1, 2015 12:52
-
-
Save ahawkins/29c09ec89cb6170cee66 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
class Scanner | |
include Concord.new(:table) | |
def scan(threads: 1) | |
results = [ ] | |
pool = (1..threads).to_a | |
workers = pool.map do |i| | |
Thread.new do | |
response = table.scan(segment: i, total_segments: threads).tap do |data| | |
results += data.items | |
end | |
while response.next_token do | |
response = table.scan({ | |
segment: i, | |
total_segments: threads, | |
next_token: response.next_token | |
}).tap do |data| | |
results += data.items | |
end | |
end | |
end | |
end | |
workers.map(&:join) | |
results | |
end | |
end |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment