CODEWITHSUNDEEP
performanceperlfile-ioruntime
user3787639
user3787639

Reputation: 89

how to improve Performance for this code?

this code is running on a file of 200M lines at least. and this takes a lot of time I would like to know if I can improve the runtime of this loop.

    my @bin_lsit; #list of 0's and 1's
    while (my $line = $input_io->getline) {
        if ($bin_list[$i]) {    
            $line =~ s/^.{3}/XXX/;
        } else {
            $line =~ s/^.{3}/YYY/;
        } 


        $output_io->appendln($line);
        $i++;
    }

Upvotes: 0

Views: 96

Answers (2)

marneborn
marneborn

Reputation: 699

Smallest change is probably to buffer between appendln's

my @bin_lsit; #list of 0's and 1's
my $i = 0;
while (my $line = $input_io->getline) {
    if ($bin_list[$i]) {    
        $line =~ s/^.{3}/XXX/;
    } else {
        $line =~ s/^.{3}/YYY/;
    } 

    $buffer .= $line;
    if ( $i % 1000 == 0 ) {
        $output_io->appendln($buffer);
        $buffer = '';
    }
    $i++;
}
if ( $buffer ne '' ) {
    $output_io->appendln($buffer);
}

Are you using IO::All? I couldn't find anything else with appendln...

Replacing this:

my $input_io = io 'tmp.this';
my $output_io = io 'tmp.out';
while (my $line = $input_io->getline ) {
    $output_io->appendln($line);
}

With this:

open(IFH, 'tmp.this');
open(OFH, '>>tmp.out');
while (my $line = <IFH> ) {
    print OFH $line;
}
close IFH;
close OFH;

Is quite a bit faster (1 sec vs 23 in my test case).

Upvotes: 0

tjd
tjd

Reputation: 4104

A regex solution may be overkill here. How about replacing the if/else blocks with:

substr($line, 0, 3, $bin_list[$i] ? 'XXX' : 'YYY';

Upvotes: 1

Related Questions