CODEWITHSUNDEEP
python
Fishtopher
Fishtopher

Reputation: 13

How to get progress of file move in python?

I've got a little script for sorting out my dowloaded files and it works great, but I'd like to print out the progress of a file move, for when it's doing the big ones, right now I do something like:

print "moving..."
os.renames(pathTofile, newName)
print "done"

But I'd like to be able to see something like a progress bar ( [..... ] style) or a percentage printed to stdout.

I don't need/want a gui of any sort, just the simplest / least-work ( :) ) way to get the operation progress).

Thanks!

Upvotes: 1

Views: 5949

Answers (3)

Joe Linoff
Joe Linoff

Reputation: 771

This example method expands on the answer by Benno by estimating the time remaining and removing the progress line when the copy is complete.

def copy_large_file(src, dst):
    '''
    Copy a large file showing progress.
    '''
    print('copying "{}" --> "{}"'.format(src, dst))

    # Start the timer and get the size.
    start = time.time()
    size = os.stat(src).st_size
    print('{} bytes'.format(size))

    # Adjust the chunk size to the input size.
    divisor = 10000  # .1%
    chunk_size = size / divisor
    while chunk_size == 0 and divisor > 0:
        divisor /= 10
        chunk_size = size / divisor
    print('chunk size is {}'.format(chunk_size))

    # Copy.
    try:
        with open(src, 'rb') as ifp:
            with open(dst, 'wb') as ofp:
                copied = 0  # bytes
                chunk = ifp.read(chunk_size)
                while chunk:
                    # Write and calculate how much has been written so far.
                    ofp.write(chunk)
                    copied += len(chunk)
                    per = 100. * float(copied) / float(size)

                    # Calculate the estimated time remaining.
                    elapsed = time.time() - start  # elapsed so far
                    avg_time_per_byte = elapsed / float(copied)
                    remaining = size - copied
                    est = remaining * avg_time_per_byte
                    est1 = size * avg_time_per_byte
                    eststr = 'rem={:>.1f}s, tot={:>.1f}s'.format(est, est1)

                    # Write out the status.
                    sys.stdout.write('\r{:>6.1f}%  {}  {} --> {} '.format(per, eststr, src, dst))
                    sys.stdout.flush()

                    # Read in the next chunk.
                    chunk = ifp.read(chunk_size)

    except IOError as obj:
        print('\nERROR: {}'.format(obj))
        sys.exit(1)

    sys.stdout.write('\r\033[K')  # clear to EOL
    elapsed = time.time() - start
    print('copied "{}" --> "{}" in {:>.1f}s"'.format(src, dst, elapsed))

You can see a fully functioning version in the gist entry here: https://gist.github.com/jlinoff/0f7b290dc4e1f58ad803.

Upvotes: 0

Greg Hewgill
Greg Hewgill

Reputation: 994531

There isn't any way to get a progress bar because the "rename" call that moves the file is a single OS call.

It's worth noting that the "rename" call only takes time if the source and destination are on different physical volumes. If they're on the same volume, then the rename will take almost no time. If you know that you're copying data between volumes, you may wish to use functions from the shutil module such as copyfileobj. There is no callback for progress monitoring, however you can implement your own source or destination file-like object to track progress.

Upvotes: 1

Benno
Benno

Reputation: 5640

You won't be able to get that kind of information using os.renames. Your best bet is to replace that with a home grown file copy operation but call stat on the file beforehand in order to get the complete size so you can track how far through you are.

Something like this:

source_size = os.stat(SOURCE_FILENAME).st_size
copied = 0
source = open(SOURCE_FILENAME, 'rb')
target = open(TARGET_FILENAME, 'wb')

while True:
    chunk = source.read(32768)
    if not chunk:
        break
    target.write(chunk)
    copied += len(chunk)
    print '\r%02d%%' % (copied * 100 / source_size),

source.close()
target.close()

Note however that this will more than likely be markedly slower than using os.rename.

Upvotes: 8

Related Questions