CODEWITHSUNDEEP
perlshellloopsdirectory
john stamos
john stamos

Reputation: 1124

Shell script to run Perl script over every directory in a directory

The setup:

A data directory that contains directories for every day of the year. ie data/2014-01-01/ to 2014-12-31. I have a perl script that I run individually inside each date directory.

I am attempting to run a shell script to run from data and go through each directory from 2014-02-15 to 2014-07-20 and run the perl script inside each directory. The perl script takes about 20 seconds to run. This is what I have so far, it will only run on February so far, and doesn't wait for the perl script to finish. I would like it to run on every directory in the range and wait for the perl script inside the loop to finish before relooping.

 #!/bin/bash

 folders=`find 2014-02*`

 for folder in $folders; do 
 cd $folder
 perl C:/Tools/script.pl
 cd ..
 done

Upvotes: 1

Views: 1026

Answers (1)

Sobrique
Sobrique

Reputation: 53478

Why not do it all in perl? It has perfectly good traversal capability with the File::Find built in module.

Encapsulate your 'script' as a subroutine. 

#!/usr/bin/perl

use strict;
use warnings;
use File::Find;

sub your_script_sub {
    my ( $dir ) = @_;
    #do something with $dir. At a worst case, you could just run your script.
    #but there's no real reason to do that, as it's perl already. 
}

sub run_script_in_dirs {
   if ( -d $File::Find::name ) { 
        your_script_sub($File::Find::name);
    }
}

find ( \&run_script_in_dirs, "/path/to/your/dir" );

For bonus points - you could use a thread to parallelise your 'run script in directory:

#!/usr/bin/perl
use strict;
use warnings;
use threads;
use Thread::Queue;

my $num_threads = 4;
my $dir_q = Thread::Queue -> new(); 

sub your_script_sub {
   while ( my $dir = $dir_q -> dequeue() ) {
          # do something in $dir;
   }
}

sub find_dirs_to_run_script {
   if ( -d $File::Find::name ) { 
        $dir_q -> enqueue($File::Find::Name);
    }
}

for ( 1..$num_threads ) {
   threads -> create ( \&your_script_sub );
}

find ( \&find_dirs_to_run_script, "/path/to/dirs" );

$dir_q -> end();

foreach my $thr ( threads -> list() ) { $thr -> join() }

Upvotes: 3

Related Questions