CODEWITHSUNDEEP
python
user3596998
user3596998

Reputation:

Copy large list of files from one location to another

(Using Python 3.4.2) I'm trying to copy a list of over 8,000 files in different locations to a staging directory. The source path would look something like this D:\dir\img0\000 and the destination path D:\stage2\NW000.tif.

From what I understand shutil can take care of this, I've found simple example on the web which I've been trying to adapt to work for my purpose. I've saved the 8,000+ records of the source and destination paths in two separate text files with line breaks after each directory path. I want the variables src and dst to store all the values of the lists and shutil.copy to copy the files to their new destination. I've been trying to test the code by creating 1 source folder and 3 dest. folders: Test, TestA, TestB, TestC. I've created two text files with a list of paths, here is how they look:

(source.txt)
C:\Users\user1\Desktop\Test\t1.txt
C:\Users\user1\Desktop\Test\t2.txt
C:\Users\user1\Desktop\Test\t3.txt

(dest.txt)
C:\Users\user1\Desktop\TestA\1t.txt
C:\Users\user1\Desktop\TestB\2t.txt
C:\Users\user1\Desktop\TestC\3t.txt

I saved the two list text files, three dummy text files and the script in the Test directory and executed the script but I didn't find the files in the destination directories. I want to know why this script isn't working (where to place the source, dest, and script files in order for the script to execute properly).:

import shutil
import errno

file = open('source.txt', 'r')
source = file.readlines()
file = open('dest.txt', 'r')
destination = file.readlines()


def copy(src, dst):
    try:
        shutil.copytree(src, dst)
    except OSError as e:
        # If the error was caused because the source wasn't a directory
        if e.errno == errno.ENOTDIR:
            shutil.copy(src, dst)
        else:
            print('Directory not copied. Error: %s' % e)

copy(source, destination)

Upvotes: 4

Views: 2765

Answers (1)

mhawke
mhawke

Reputation: 87064

shutil.copytree(src, dst) copies one directory tree from src to dst. It does not automatically iterate over lists of files. You could try zipping source and destination lists and looping of the resultant list:

for src, dst in zip(source, destination):
    copy(src, dst)

You might also run into problems when you fallback to shutil.copy(src, dst) - this will fail if the destination directory does not already exist.

Upvotes: 5

Related Questions