CODEWITHSUNDEEP
pythonmultiprocessingtqdm
Ad Blu
Ad Blu

Reputation: 41

TQDM and multiprocessing - python

I want to use 4 python processes and tqdm progress bar in each of them:

 66%|███████████████████████████▌              | 80/122 [00:44<00:23,  1.79it/s]
 35%|██████████████▉                           |  18/52 [00:08<00:16,  2.01it/s]
 26%|█████████▋                                | 61/108 [00:44<00:34,  1.38it/s]
 56%|███████████████████████▋                  | 61/108 [00:44<00:34,  1.38it/s]

so they are all 4 displayed and updated like one at the same time. How can I achieve that ?

Upvotes: 3

Views: 5404

Answers (1)

D Hudson
D Hudson

Reputation: 1092

import multiprocessing
import random
import time

import tqdm


def task(position, lock):
    total_iterations = 5

    with lock:
        bar = tqdm.tqdm(
            desc=f'Position {position}',
            total=total_iterations,
            position=position,
            leave=False
        )

    for _ in range(total_iterations):
        time.sleep(random.uniform(0.5, 1))
        with lock:
            bar.update(1)

    with lock:
        bar.close()


if __name__ == '__main__':
    lock = multiprocessing.Manager().Lock()

    with multiprocessing.Pool(4) as pool:
        for position in range(4):
            pool.apply_async(task, args = (position + 1, lock))

        pool.close()
        pool.join()

Explanation

  • Using tqdm's position flag (as suggested in the comments), I set the line number to print each bar on.
  • I use a Lock instance to prevent multiple processes writing to stdout at the same time.

Upvotes: 12

Related Questions