platypus.mpipool module

class platypus.mpipool.MPIPool(comm=None, debug=False, loadbalance=False)

A pool that distributes tasks over a set of MPI processes using mpi4py. MPI is an API for distributed memory parallelism, used by large cluster computers. This class provides a similar interface to Python’s multiprocessing Pool, but currently only supports the map() method.

Contributed initially by Joe Zuntz.

Parameters

comm(optional)

The mpi4py communicator.

debugbool (optional)

If True, print out a lot of status updates at each step.

loadbalancebool (optional)

if True and the number of taskes is greater than the number of processes, tries to loadbalance by sending out one task to each cpu first and then sending out the rest as the cpus get done.

bcast(*args, **kwargs)

Equivalent to mpi4py bcast() collective operation.

close()

Just send a message off to all the pool members which contains the special _close_pool_message sentinel.

is_master()

Is the current process the master?

map(function, tasks, callback=None)

Like the built-in map() function, apply a function to all of the values in a list and return the list of results.

Parameters

functioncallable

The function to apply to each element in the list.

tasks :

A list of tasks – each element is passed to the input function.

callbackcallable (optional)

A callback function to call on each result.

wait()

If this isn’t the master process, wait for instructions.

exception platypus.mpipool.MPIPoolException(tb)