copy specific files from multiple directories using python

Question

I'm quite new to python, and I am stucked with a piece of code I'd like to achieve. I hope I'll be to find some help here.

I have a specific list of files I'd like to copy in one directory. I've read lot of advice on the Internet but I haven't read something specific enough for me.

So, here's the situation : I have multiple pictures files in different folders in "FROM" folder. I'd like to copy only the ones I'd like into a destination folder "TO"

For the moment, here is what I have (from another topic on Stackoverflow) :

src = "/Volumes/MacintoshHD/TEST_SHUTIL/FROM"
dst = "/Volumes/MacintoshHD/TEST_SHUTIL/TO"

file_names = ["120099.TIFF", "901664.TIFF", "902257.TIFF"]

for file_name in file_names:
    full_file_name = os.path.join(src, file_name)
    if (os.path.isfile(full_file_name)):
        shutil.copy(full_file_name, dst)

I wanted to know if there was a way to find the files inside subfolders like a isfile but without testing every folders.

I looked at copytree, but it doesn't look very useful for me...

At first I wanted to create a dictionnary wih the file and its url, but it's more complicated than expected (for many reasons), so the easiest way would be to be able to find those files in subfolders.

Many Tanks

Bart.

Answer 1

Thank you every one for your answers. It's been a while since I posted my question, and I hadn't any time to test your answers since. So please excuse me for this.

I have read them carefully. I think I understood them, even if they didn't work like this.

I finally manage to get what I want but it's note as smart as it could be.

I got the copy of files by creating a list of folders.

Here's the code :

import os, shutil, fnmatch

src = ["/Volumes/MacintoshHD/TEST_SHUTIL/FROM/1", "/Volumes/MacintoshHD/TEST_SHUTIL/FROM/2", "/Volumes/MacintoshHD/TEST_SHUTIL/FROM/3"]
dst = "/Volumes/MacintoshHD/TEST_SHUTIL/TO"

listePlanDev = ["00120099.TIFF", "06901664.TIFF", "06902257.TIFF"]

for file_name in listePlanDev :
    for dir_names in src:
        full_file_name = os.path.join(dir_names, file_name)
        if (os.path.isfile(full_file_name)):
            shutil.copy(full_file_name, dst)

So here is the tree I have at he beginning :

\Volumes
    \MacintoshHD
        \TEST_SHUTIL
            \FROM
                \1
                - 00100001.TIFF
                - 00120099.TIFF
                \2
                - 06900001.TIFF
                - 06901664.TIFF
                \3
                - 06902257.TIFF
            \TO

And at the end I'd like

\Volumes
    \MacintoshHD
        \TEST_SHUTIL
            \FROM
                \1
                - 00100001.TIFF
                - 00120099.TIFF
                \2
                - 06900001.TIFF
                - 06901664.TIFF
                \3
                - 06902257.TIFF
            \TO
            - 120099.TIFF
            - 901664.TIFF
            - 902257.TIFF

Now I need to create the list of directories in FROM with os.walk I guess. Beacuase there should be thousands of them when I'll use program.

This makes a lot of stages and I'm sure there's a better way to achieve it in less lines of code...

Anyway, I wanted to give a "feedback" to your answers, and thank you all.

Bart.

Answer 2

This will make a list containing the source directory's file names and its subdirectories' file names all in one:

file_names = [os.path.join(dp, f) for dp, dn, fn 
                     in os.walk(src_dir) for f in fn]

then you can continue with your code:

for file_name in file_names:
    full_file_name = os.path.join(src, file_name)
    if (os.path.isfile(full_file_name)):
        shutil.copy(full_file_name, dst)

Though you have to search through the entire directory tree, but only once.

Answer 3

Maybe use Unix's find command inside Python to search for all .TIFF files inside the base folder?

import os
file_paths = os.popen('find . -name \*.TIFF -print').read().split('\n')

Here . signifies current directory, you can change this to whatever path you wish to use as base directory.

Output

>> for each in file_path:
       print each
 /Cpp/seive.TFF
 /Cpp/seive2.TFF
 ....

Answer 4

Using os.walk and fnmatch:

copytiffs.py

#~/usr/bin/env python
import os, shutil, fnmatch

def main():
    src = "Volumes/MacintoshHD/TEST_SHUTIL/FROM"
    dst = "Volumes/MacintoshHD/TEST_SHUTIL/TO"
    file_names = ["120099.TIFF", "901664.TIFF", "902257.TIFF"]

    for root, dirnames, filenames in os.walk(src):
        for target in file_names:
            for candidate_filename in fnmatch.filter(filenames, target):
                shutil.copy(os.path.join(root, candidate_filename), dst)

if __name__ == '__main__':
    main()

BEFORE RUNNING SCRIPT*

[root@joeyoung.io copyfiles]# tree ./
./
`-- Volumes
    `-- MacintoshHD
        `-- TEST_SHUTIL
            |-- FROM
            |   |-- 120099.TIFF
            |   |-- ignoreme.txt
            |   |-- sub1
            |   |   |-- 901664.TIFF
            |   |   `-- ignoreme.txt
            |   `-- sub2
            |       |-- 902257.TIFF
            |       `-- ignoreme.txt
            `-- TO

RUNNING SCRIPT

[root@joeyoung.io copyfiles]# python copytiffs.py

AFTER RUNNING SCRIPT

[root@joeyoung.io copyfiles]# tree
.
|-- Volumes
|   `-- MacintoshHD
|       `-- TEST_SHUTIL
|           |-- FROM
|           |   |-- 120099.TIFF
|           |   |-- ignoreme.txt
|           |   |-- sub1
|           |   |   |-- 901664.TIFF
|           |   |   `-- ignoreme.txt
|           |   `-- sub2
|           |       |-- 902257.TIFF
|           |       `-- ignoreme.txt
|           `-- TO
|               |-- 120099.TIFF
|               |-- 901664.TIFF
|               `-- 902257.TIFF
`-- copytiffs.py