Skip to content

Instantly share code, notes, and snippets.

@ajtritt
Created October 6, 2017 19:39
Show Gist options
  • Save ajtritt/2e064f07a9f4ef3993dfb6223749c105 to your computer and use it in GitHub Desktop.
Save ajtritt/2e064f07a9f4ef3993dfb6223749c105 to your computer and use it in GitHub Desktop.
A script to demonstrate how parallelize embarassingly parallel problems using core Python functionality
import filterframework
import pynwb
import multiprocessing # this is a core Python package
def my_analysis_func(...):
# user writes this
...
nwb_file = pynwb.NWBFile(...)
query_string = 'a toy query string'
query_result = filterframeork.get_data_from_query(nwb_file, query_string)
num_cores = 4
pool = multiprocessing.Pool(num_cores)
analyzed_query_results = pool.map(my_analysis_func, query_result)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment