Skip to content

Instantly share code, notes, and snippets.

@Ch4s3
Last active June 7, 2016 03:54
Show Gist options
  • Save Ch4s3/22f53ecc7c9c69a7fc718f08dd5326e1 to your computer and use it in GitHub Desktop.
Save Ch4s3/22f53ecc7c9c69a7fc718f08dd5326e1 to your computer and use it in GitHub Desktop.
require "nokogiri"
require "concurrent"
################################
# Methods
################################
def parse_data_into_chuncks(data)
chunks = {}
ccda.css("chunk section").each do |chunk|
chunk_name = component.at_css("foo[bar='Chunk']")["chunkName"]
chunks[chunk_name] = chunk
end
chunks
end
def scrape_chunk_a(xml) do
#implement parsing
end
def scrape_chunk_b(xml) do
#implement parsing
end
def scrape_chunk_b(xml) do
#implement parsing
end
################################
# Important bits
################################
xml_data = Nokogiri::XML.parse(data_payload) { |config| config.noblanks }
xml_data.remove_namespaces! #note this bang method strips namespaces in place, so its faster, but unsafe
parsed_chunks = parse_data_into_chunck(xml_data)
promise_a = Concurrent::Promise.new do
scrape_chunk_a(parsed_chunks["The Name of Chunk A from `parse_data_into_chuncks`"])
end
promise_b = Concurrent::Promise.new do
scrape_chunk_b(parsed_chunks["The Name of Chunk B from `parse_data_into_chuncks`"])
end
promise_c = Concurrent::Promise.new do
scrape_chunk_c(parsed_chunks["The Name of Chunk C from `parse_data_into_chuncks`"])
end
promises_hash = {
a: promise_a,
b: promise_b,
c: promise_c
}
promises = promises_hash.values
all_promises = = Concurrent::Promise.all?(*promises).then do
{
a: a.value(1)[:some_thing],
b: b.value(1)[:some_other_thing],
c: d.value(1)[:the_last_thing]
}
end
all_promises.execute
final_data = all_promises.value(3) #or however long you need to wait
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment