How to use with a real function?

[daviddoria]

This is nice, but with a non-toy function, you would want to pass some actual data to every call of test(), not just the location to display the progress bar, right? As I understand it you can only pass one variable thing (and as many constant things you want via a Partial) to the map() calls.

[aaren]

You can pass a single list or dictionary containing whatever data you want.

def work_function(data):
    time.sleep(data)

def test_function(payload):
    writer = Writer(payload['location'])
    pbar = ProgressBar(fd=writer)
    pbar.start()
    for i in range(100):
        # mimic doing some stuff
        work_function(payload['data'])
        pbar.update(i)
    pbar.finish()

locations = [(0, 1), (0, 6), (0, 7)]
data = [0.01, 0.02, 0.03]

payload = [{'location': loc, 'data': d} for loc, d in zip(locations, data)]

p = Pool()
p.map(test_function, payload)
p.close()

If you want more control than Pool can give you then you could start up a number of Process instances.

https://docs.python.org/2/library/multiprocessing.html