CODEWITHSUNDEEP
linuxshellscripting
user1846251
user1846251

Reputation: 45

Loop through subfolders and files in a Linear fashion

I need to loop through files in subfolders in the below fashion and copy to remote directory and my source directory looks like this

 **Parent_Folder        Folder 1/            COMP1/          file1.txt file2.txt
                                           COMP2/          file1.txt file2.txt
                                           COMP3/          file1.txt file2.txt
                        
                         Folder 2/         COMP1/          file1.txt file2.txt
                                           COMP2/          file1.txt file2.txt
                                           COMP3/          file1.txt file2.txt


                         Folder 3/         COMP1/          file1.txt file2.txt
                                           COMP2/          file1.txt file2.txt
                                           COMP3/          file1.txt file2.txt**


I need to iteratively loop through files and copy to target folder in this manner from shell-script

    1.) Parent_Folder/Folder 1/Comp1/file1.txt 
    2.) Parent_Folder/Folder 1/Comp1/file2.txt 
    3.) Parent_Folder/Folder 1/Comp2/file1.txt 
    4.) Parent_Folder/Folder 1/Comp2/file2.txt 
    5.) Parent_Folder/Folder 1/Comp2/file1.txt 
    6.) Parent_Folder/Folder 1/Comp2/file2.txt 
        
    Target Folder / Comp1 / file.txt
                    Comp1 / file2.txt
                    Comp2 / file1.txt
                    Comp2 / file2.txt
                    Comp3 / file1.txt
                    Comp3 / file3.txt 
      
 

    // Do some processing on this new copied files ..
          
    7.) Parent_Folder/Folder 2/Comp1/file1.txt 
    8.) Parent_Folder/Folder 2/Comp1/file2.txt 
    9.) Parent_Folder/Folder 2/Comp2/file1.txt 
    10.) Parent_Folder/Folder 2/Comp2/file2.txt 
    11.) Parent_Folder/Folder 2/Comp3/file1.txt 
    12.) Parent_Folder/Folder 2/Comp3/file2.txt 

    Target Folder / Comp1 / file.txt
                    Comp1 / file2.txt
                    Comp2 / file1.txt
                    Comp2 / file2.txt
                    Comp3 / file1.txt
                    Comp3 / file3.txt 
   
// Do some processing on this new copied files ..
    13.) Parent_Folder/Folder 3/Comp1/file1.txt 
    14.) Parent_Folder/Folder 3/Comp1/file2.txt 
    15.) Parent_Folder/Folder 3/Comp2/file1.txt 
    16.) Parent_Folder/Folder 3/Comp2/file2.txt 
    17.) Parent_Folder/Folder 3/Comp3/file1.txt 
    18.) Parent_Folder/Folder 3/Comp3/file2.txt 

    Target Folder      / Comp1 / file.txt
                        Comp1 / file2.txt
                        Comp2 / file1.txt
                        Comp2 / file2.txt
                        Comp3 / file1.txt
                        Comp3 / file3.txt

I am able to iterate through a simple folder with for file in Parent_folder/*; , but I dont think this serves my purpose. Any help is appreciated.

In a nutshell, copy the parent folder sub-folders COMP1, COMP2, COMP3 to target folder COMP1, COMP2, COMP3..

Upvotes: 0

Views: 24

Answers (1)

Nicholas Rees
Nicholas Rees

Reputation: 753

Previous comment about rsync is spot on. Something like this should work.

rsync -av Parent_Folder/Folder\ 1/ /path/to/Target/
# do some processing
rm -rf /path/to/Target/*  # do this to avoid ending up with mixture of Folder 1 + folder 2 data
rsync -av Parent_Folder/Folder\ 2/ /path/to/Target/
# do some processing
rm -rf /path/to/Target/*  # do this to avoid ending up with mixture of Folder 2 + folder 3 data
rsync -av Parent_Folder/Folder\ 3/ /path/to/Target/
# do some processing
rm -rf /path/to/Target/*  # do this to avoid ending up with mixture of Folder 3 + folder 1 data on next run

Upvotes: 1

Related Questions