Created
October 6, 2017 19:39
-
-
Save ajtritt/2e064f07a9f4ef3993dfb6223749c105 to your computer and use it in GitHub Desktop.
A script to demonstrate how parallelize embarassingly parallel problems using core Python functionality
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import filterframework | |
import pynwb | |
import multiprocessing # this is a core Python package | |
def my_analysis_func(...): | |
# user writes this | |
... | |
nwb_file = pynwb.NWBFile(...) | |
query_string = 'a toy query string' | |
query_result = filterframeork.get_data_from_query(nwb_file, query_string) | |
num_cores = 4 | |
pool = multiprocessing.Pool(num_cores) | |
analyzed_query_results = pool.map(my_analysis_func, query_result) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment