Skip to content

Instantly share code, notes, and snippets.

@ahawkins
Created September 1, 2015 12:52
Show Gist options
  • Save ahawkins/29c09ec89cb6170cee66 to your computer and use it in GitHub Desktop.
Save ahawkins/29c09ec89cb6170cee66 to your computer and use it in GitHub Desktop.
class Scanner
include Concord.new(:table)
def scan(threads: 1)
results = [ ]
pool = (1..threads).to_a
workers = pool.map do |i|
Thread.new do
response = table.scan(segment: i, total_segments: threads).tap do |data|
results += data.items
end
while response.next_token do
response = table.scan({
segment: i,
total_segments: threads,
next_token: response.next_token
}).tap do |data|
results += data.items
end
end
end
end
workers.map(&:join)
results
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment