Skip to content

Instantly share code, notes, and snippets.

@poros
Created October 4, 2015 18:20
Show Gist options
  • Save poros/9afff244461b55b29883 to your computer and use it in GitHub Desktop.
Save poros/9afff244461b55b29883 to your computer and use it in GitHub Desktop.
Process/thread pool for embarrassingly parallel problems
from multiprocessing import Pool
pool = Pool(4)
results = pool.map(urllib2.urlopen, urls)
pool.close()
pool.join()
# if you want threads instead of processes
from multiprocessing.dummy import Pool
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment