CODEWITHSUNDEEP
pythontqdm
Nicolas Gervais
Nicolas Gervais

Reputation: 36684

Is it possible to use tqdm for a process that isn't a loop?

I am writing a program in which an operation takes a few seconds (loading a large file). Luckily, it always takes the same amount of time. So, for the sake of the user, I want to make a progress bar. However, tqdm seems to be designed for loops. Let's say that sleep(10) is the process that takes time. How do I work this out?

I want to make a progress bar for a long process that is not a loop.

from time import time, sleep
from tqdm import tqdm

for i in tqdm([1]):
    sleep(10)

The problem with this code is that the progress bar will stay at zero, then jump to 100% at the end of the process. I want a progress bar that evolves consistently over the course of 10 seconds.

Upvotes: 12

Views: 18039

Answers (4)

Jeremiah Horstick
Jeremiah Horstick

Reputation: 151

Manually set the progress value for tqdm:

#Manually set p_bar
p_bar = tqdm(range(10))
p_bar.update(5)
p_bar.refresh()
time.sleep(1)
#another way to do it
p_bar.n = 7
p_bar.refresh()

Upvotes: 12

Jose Antonio Martin H
Jose Antonio Martin H

Reputation: 1511

for _ in tqdm(range(10),desc="waiting..."):
    time.sleep(1)

Upvotes: 2

DavidBu
DavidBu

Reputation: 526

You can use a thread to load the file, and loop through tqdm(range(10)) until the file is loaded like this:

import logging
import threading
import time
from tqdm import tqdm


def thread_function(name):
    logging.info("Thread %s: starting", name)
    logging.info("Main    : file loading...")
    time.sleep(5)
    logging.info("Thread %s: finishing", name)


if __name__ == "__main__":
    logging.basicConfig(format="%(asctime)s: %(message)s", level=logging.INFO,
                        datefmt="%H:%M:%S")

    x = threading.Thread(target=thread_function, args=(1,))
    x.start()
    for i in tqdm(range(10)):
        if not x.is_alive():
            break
        time.sleep(1)
    x.join()
    logging.info("Main    : end.")

Upvotes: 2

CarlosSR
CarlosSR

Reputation: 1195

You can define a chunk size. A bar that lasts 10 seconds assumes you already know the time it takes to load the file...

Try this

from tqdm import tqdm_notebook as tqdm

chunk_size = 1000
total_chunks=nrows/chunk_size

chunks = pd.read_sql_query(query, connection, index_col='index_name',chunksize= chunk_size)
raw_train_data=pd.DataFrame()

with tqdm(total=total_chunks) as pbar:

    for chunk in chunks:
        raw_train_data = pd.concat([raw_train_data, chunk])
        pbar.update(1)

Upvotes: 4

Related Questions