CODEWITHSUNDEEP
pythonloggingprintingtqdm
FluidMechanics Potential Flows
FluidMechanics Potential Flows

Reputation: 594

tqdm and print / logging doesn't work properly (no linebreak for first logging / print statement)

Here is a minimal reproducible example:

import logging
from tqdm import tqdm

for i in tqdm(range(1000)):
    for _ in range(2000):
        a = i+8
    if i % 100 == 0:
        logging.warning("test")

This outputs this:

  0%|          | 0/1000 [00:00<?, ?it/s]WARNING:root:test
WARNING:root:test
WARNING:root:test
WARNING:root:test
WARNING:root:test
WARNING:root:test
WARNING:root:test
WARNING:root:test
WARNING:root:test
WARNING:root:test
100%|██████████| 1000/1000 [00:00<00:00, 996982.17it/s]

I'm fine with the bar refreshing and printing again (without the argument leave=True basically) but there is no linebreak for the first warning. It doesn't work very well either with a print("test"):

  0%|          | 0/1000 [00:00<?, ?it/s]test
test
test
test
test
test
test
test
test
test
100%|██████████| 1000/1000 [00:00<00:00, 20526.40it/s]

Any idea how to have a proper linebreak for the first print / logging? I can't add "\n" because otherwise i'm gonna have way too many linebreaks for the following print messages. And adding a "if this is the first message then add "\n" otherwise don't" sounds like a dirty way of bypassing the problem.

I've seen this post (tqdm with logger on separate lines) but this was no answers apart from "do your own tqdm" which i don't really like because i like just having to do tqdm(my_iterable).

Upvotes: 3

Views: 1489

Answers (1)

blhsing
blhsing

Reputation: 106995

You can clear the status bar by calling the clear method of the tqdm object before writing output to stdout or stderr.

To make the call to the clear method happen automatically for each output, you can create a context manager that patches both sys.stdout and sys.stderr with a wrapper function that calls the clear method of a given tqdm object before writing a message to the original sys.stderr:

import sys
from unittest.mock import patch
from contextlib import contextmanager

@contextmanager
def tqdm_output(tqdm, write=sys.stderr.write):
    def wrapper(message):
        if message != '\n':
            tqdm.clear()
        write(message)
        if '\n' in message:
            tqdm.display()

    with patch('sys.stdout', sys.stderr), patch('sys.stderr.write', wrapper):
        yield tqdm

So that you can put the iteration within the context manager and use the logger normally:

with tqdm_output(tqdm(range(1000))) as trange:
    for i in trange:
        if i % 100 == 0:
            logging.warning("test")

And it would output:

WARNING:root:test                                                        
WARNING:root:test
WARNING:root:test                                                        
WARNING:root:test
WARNING:root:test                                                        
WARNING:root:test
WARNING:root:test
WARNING:root:test                                                        
WARNING:root:test
WARNING:root:test                                                        
100%|██████████████████████████████| 1000/1000 [00:00<00:00, 1797.66it/s]

Same with using print:

with tqdm_output(tqdm(range(1000))) as trange:
    for i in trange:
        if i % 100 == 0:
            print('test')

which outputs:

test                                                                     
test                                                                     
test                                                                     
test                                                                     
test                                                                     
test                                                                     
test                                                                     
test                                                                     
test                                                                     
test                                                                     
100%|███████████████████████████████| 1000/1000 [00:05<00:00, 181.55it/s]

Demo: https://replit.com/@blhsing/GentleShallowScan

Upvotes: 2

Related Questions