Skip to content

Instantly share code, notes, and snippets.

@ZGainsforth
Created July 2, 2015 19:31
Show Gist options
  • Select an option

  • Save ZGainsforth/63f0e543a4a900276f3f to your computer and use it in GitHub Desktop.

Select an option

Save ZGainsforth/63f0e543a4a900276f3f to your computer and use it in GitHub Desktop.
Multiprocessing pool example to use all your CPU cores on a computationally intensive stack using a 3D numpy array as input.
from multiprocessing import Pool
import numpy as np
n = 1000000
x = 5
y = 5
z = 100
def f(x):
#print x.shape
for i in range(n):
x = x**2
x = np.sqrt(x)
return np.sum(x)
if __name__ == '__main__':
m = np.random.random((x,y,z))
#print m[:,:].shape
result = np.ndarray((x,y), dtype=object)
p = Pool()
for i in range(m.shape[0]):
for j in range(m.shape[1]):
result[i,j] = p.apply_async(f, [m[i,j,:]])
p.close()
numresult = np.zeros((x,y))
for i in range(m.shape[0]):
for j in range(m.shape[1]):
numresult[i,j] = result[i,j].get()
print numresult
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment