CODEWITHSUNDEEP
pythonprogress-bartqdm
Tom de Geus
Tom de Geus

Reputation: 5975

tqdm: simple loop on iterations, show MB/s

Question

I want to do a simple loop over n iterations. Each iteration contains a copy operation of which I know how many bytes were copied. My question is: how can I also show the number of MB/s in the progress bar?

Example

I'm showing a progress bar around rsync. I modified this answer as follows:

import subprocess
import sys
import re
import tqdm

n = len(open('myfiles.txt').readlines())
your_command = 'rsync -aP --files-from="myfiles.txt" myhost:mysource .'
pbar = tqdm.trange(n)

process = subprocess.Popen(your_command, stdout=subprocess.PIPE, shell=True)
for line in iter(process.stdout.readline, ''):
    line = line.decode("utf-8")
    if re.match(r'(.*)(xfr\#)([0-9])(.*)(to\-chk\=)([0-9])(.*)', line):
        pbar.update()

Whereby myfiles.txt contains a list of files. This gives me a good progress bar, showing the number of iterations per second.

However, the summary line that I match on to signal that a file was copied, e.g.

  682,356 100%  496.92kB/s    0:00:01 (xfr#5, to-chk=16756/22445)

also contains the number of bytes that were copied, which I want to use to show a copying speed.

Upvotes: 2

Views: 2823

Answers (1)

Arty
Arty

Reputation: 16757

Below I provided code doing what you need.

Because I didn't have your example data I created simple generator function that emulates process of retrieving data. It generates at random points of time (on each iteration) value equal to random number of bytes (not mega-bytes).

There are two tqdm bars below, one is used for progress, it measures iterations per second and total number of iterations done and percentage. And second bar that measures speed in megabytes per second and total number of megabytes received.

Try it online!

import tqdm
def gen(cnt):
    import time, random
    for i in range(cnt):
        time.sleep(random.random() * 0.125)
        yield random.randrange(1 << 20)
total_iterations = 150
pbar = tqdm.tqdm(total = total_iterations, ascii = True)
sbar = tqdm.tqdm(unit = 'B', ascii = True, unit_scale = True)
for e in gen(total_iterations):
    pbar.update()
    sbar.update(e)

Output:

  9%|█████████▉                               | 89/1000 [00:11<01:53,  8.05it/s]
40.196MiB [00:11,  3.63MiB/s]

ASCII video (+link):

gif

Upvotes: 5

Related Questions