Multiprocessing example

README.md

Introduction

This is an example that shows the differences in multiprocessing execution between POSIX systems (where fork() is available) and Windows (where multiprocessing does some magic to recreate the Python state in spawned interpreters, namely, reimporting the main module as well as the one where the function lives). It also shows that environment variable are inherited.

Results

On Windows (no fork())

main running as __main__
func running
other running
main running again
main running again
func running
foo: '42'
foo: '42'
res = [2, 4]

On GNU/Linux (with fork())

main running as __main__
func running
other running
foo: '42'
foo: '42'
res = [2, 4]

func.py

import os

print "func running"

def foo(a):
    print "foo: %r" % os.getenv('BLABLA')
    return a*2

main.py

if __name__ == '__main__':
    print "main running as __main__"
    import other
    other.do_stuff()
else:
    print "main running again"

other.py

import multiprocessing
import os

from func import foo


print "other running"

def do_stuff():
    os.environ['BLABLA'] = '42'
    pool = multiprocessing.Pool(2)
    res = pool.map(foo, [1, 2])
    pool.close()
    pool.join()
    print "res = %r" % (res,)