Capturing command-line output on win32 that hasn't been flushed yet

Question

(Context: I'm trying to monitor a long-running process from a Perl CGI script. It backs up an MSSQL database and then 7-zips it. So far, the backup part (using WITH STATS=1) outputs to a file, which I can have the browser look at, refreshing every few seconds, and it works.)

I'm trying to use 7zip's command-line utility but capture the progress bar to a file. Unfortunately, unlike SQL backups, where every time another percent is done it outputs another line, 7zip rewinds its output before outputting the new progress data, so that it looks nicer if you're just using it normally on the command-line. The reason this is unfortunate is that normal redirects using >, 1>, and 2> only create a blank file, and no output ever appears in it, except for >, which has no output until the job is done, which isn't very useful for a progress bar.

How can I capture this kind of output, either by having every change in % somehow be appended to a logfile (so I can use my existing method of logfile monitoring) just using command-line trickery (no Perl), or by using some Perl code to capture it directly after calling system()?

Answer 1

This doesn't answer how to capture output that gets rewound, but it was a useful way of going about it that I ended up using.

For restores:

1. use 7za l to list the files in the zip file and their sizes
2. fork 7za e using open my $command
3. track each file as it comes out with -s $filename and compare to the listing
4. when all output files are their full size, you're done

For backups:

1. create a unique dir somewhere
2. fork 7za a -w
3. find the .tmp file in the dir
4. track its size
5. when the .tmp file no longer exists, you're done

For restores you get enough data to show a percentage done, but for backups you can only show the total file size so far, but you could compare with historical ratios if you're using similar data to get a guestimate. Still, it's more feedback than before (none).

Answer 2

If you need to capture the output all at once then this is the code you want:

$var=`echo cmd`;

If you want to read the output line by line then you need this code:

#! perl -slw
use strict;
use threads qw[ yield async ];
use threads::shared;

my( $cmd, $file ) = @ARGV;
my $done : shared = 0;
my @lines : shared;

async {
    my $pid = open my $CMD, "$cmd |" or die "$cmd : $!";
    open my $fh, '>', $file or die "$file : $!";
    while( <$CMD> ) {
        chomp;
        print $fh $_;         ## output to the file
        push @lines, $_;    ## and push it to a shared array
    }
    $done = 1;
}->detach;

my $n = 0;
while( !$done ) {
    if( @lines ) {            ## lines to be processed
        print pop @lines;   ## process them
    }
    else {
        ## Else nothing to do but wait.
        yield;
    }
}

Another option is using Windows create process. I know Windows C/C++ create process will allow you to redirect all stdout. Perl has access to this same API call: See Win32::Process.

Answer 3

You can try opening a pipe to read 7zip's output.