CODEWITHSUNDEEP
pythonpandas
arqchicago
arqchicago

Reputation: 309

pandas read_csv method busy status

I am trying to implement a "busy" status indicator when using Python Pandas read_csv method to upload data from csv files. This is particularly useful when uploading large csv files that keep the terminal busy until data is loaded into the memory. However, I can't get any simple solution to work with Pandas. A simple cursor indicator such as this one that I use would be very useful if it can be integrated in the read_csv method.

import itertools
import sys

busy = itertools.cycle([' \ ', ' / '])
sys.stdout.write(next(busy))
sys.stdout.flush()
sys.stdout.write('\b\b\b')

Any suggestions on implementing such a solution for the read_csv method in pandas?

Upvotes: 0

Views: 155

Answers (1)

user2668284
user2668284

Reputation:

You could use some kind of "monitor" thread. Something like this:

import itertools
from threading import Thread
import time
import sys

BUSY = True

def busy():
    b = itertools.cycle([' \ ', ' / '])
    while BUSY:
        sys.stdout.write(next(b))
        sys.stdout.flush()
        sys.stdout.write('\b\b\b')
        time.sleep(0.2)

t = Thread(target=busy)
t.start()
# sleep here emulates your time-consuming operation
time.sleep(5)

BUSY = False
t.join()

Your could also consider using events. I offer this only as a very simplistic approach.

EDIT: Here is a multiprocessing approach:

from multiprocessing import Pool, Manager, freeze_support
import sys
import time

def busy(v):
    flash = [' | ', ' / ', ' - ',  ' \\ ', ' | ', ' / ', ' - ', ' \\ ']
    fi = 0
    while v.value > 0:
        sys.stdout.write(flash[fi])
        sys.stdout.flush()
        sys.stdout.write('\b\b\b')
        time.sleep(0.25)
        fi = (fi + 1) % len(flash)

def main():
    with Manager() as manager:
        v = manager.Value('i', 1)
        with Pool() as pool:
            ar = pool.apply_async(busy, [v])
            time.sleep(5) # emulate some time-consuming process
            v.value = 0
            ar.get()
 

if __name__ == '__main__':
    freeze_support()
    main()

Upvotes: 4

Related Questions