Python multiprocessing hello world. Split a list and process sublists in different jobs

hello_multiprocessing.py

import multiprocessing

# split a list into evenly sized chunks
def chunks(l, n):
    return [l[i:i+n] for i in range(0, len(l), n)]


def do_job(job_id, data_slice):
    for item in data_slice:
        print "job", job_id, item


def dispatch_jobs(data, job_number):
    total = len(data)
    chunk_size = total / job_number
    slice = chunks(data, chunk_size)
    jobs = []

    for i, s in enumerate(slice):
        j = multiprocessing.Process(target=do_job, args=(i, s))
        jobs.append(j)
    for j in jobs:
        j.start()


if __name__ == "__main__":
    data = ['a', 'b', 'c', 'd']
    dispatch_jobs(data, 2)

very elegant. thanks for sharing!

As someone just learning to program, this finally made multiprocessing make sense. Thanks for posting.

That is a nice example, thank you!

Exactly what I needed to see. Thank you.

Good example for the beginner programming.

For python 3.7, the section
slice = chunks(data, chunk_size)

needs to be changed to:
slice = chunks(data, int(chunk_size))

To avoid:
TypeError: 'float' object cannot be interpreted as an integer

Good example for the beginner programming.

For python 3.7, the section
slice = chunks(data, chunk_size)

needs to be changed to:
slice = chunks(data, int(chunk_size))

To avoid:
TypeError: 'float' object cannot be interpreted as an integer

or
chunk_size = total // job_number

it works to me! thanks