How to Perl calculate array column data?

Question

Here is full code:

foreach my $file (@safiles) {
    @sa_data = qx(/usr/sbin/sa -m $file);
       foreach my $user (@cplist) {
       push(@all, grep { /$user/ } @sa_data);
  }
}

foreach  my $user (@cplist) {
    @data = grep { /$user/ } @all;
    print @data;
    print ".............\n";
}

I am looping through system acct data file & collecting lines for each system users and pushing it to an array called @all.

Then I am grepping each user on @all & sending it to @data array. If I print @data it shows each user's data in batches:

cp1556                                1       0.01re       0.01cp         0avio     22336k
cp1556                                1       0.01re       0.01cp         0avio     22336k
cp1556                               63     862.28re       0.17cp         0avio     13839k
cp1556                               43     176.03re       0.13cp         0avio     12083k
cp1556                               40     653.49re       0.12cp         0avio     13258k
cp1556                               30     506.61re       0.05cp         0avio     12177k
cp1556                                4       0.02re       0.02cp         0avio     19736k
.............
cp1449                                  6       0.26re       0.11cp         0avio     30176k
cp1449                                  3       0.07re       0.04cp         0avio     30261k
cp1449                                  2       0.00re       0.00cp         0avio     17135k

Now I want to sum the columns here where output == cp1449 $2total, $3total, $4total etc

Answer 1

You can move filtering and summing up the input per user into a single loop. No need to iterate twice.

use strict;
use warnings;
use Data::Dumper;

my @cplist = qw(cp1556 cp1449); # users
my @sa_data = <DATA>; # qx()

my %wanted_users = map { $_ => 1 } @cplist; # build a lookup table
my %sums;
foreach my $line (@sa_data) {
    # break into colums
    my @cols = split /\s+/, $line;

    # do we want to collect data for this user?
    if (exists $wanted_users{$cols[0]}) {
        for my $i ( 1, 2, 3, 5 ) {
            ( my $number = $cols[$i] ) =~ s/[^\d.]//g;
            $sums{$cols[0]}{$i} += $number;
        }
    }
}

print Dumper \%sums;

__DATA__
cp1556                                1       0.01re       0.01cp         0avio     22336k
cp1556                                1       0.01re       0.01cp         0avio     22336k
cp1556                               63     862.28re       0.17cp         0avio     13839k
cp1556                               43     176.03re       0.13cp         0avio     12083k
cp1556                               40     653.49re       0.12cp         0avio     13258k
cp1556                               30     506.61re       0.05cp         0avio     12177k
cp1556                                4       0.02re       0.02cp         0avio     19736k
.............
cp1449                                  6       0.26re       0.11cp         0avio     30176k
cp1449                                  3       0.07re       0.04cp         0avio     30261k
cp1449                                  2       0.00re       0.00cp         0avio     17135k

This code reads from the DATA filehandle instead of your qx().

I've created a lookup hash for the users we are interested in, which come from @cplist. Checking if a hash key exists is much cheaper than doing the pattern match and storing an extra array with all the lines.

We then split your lines of data into columns on as much whitespace as the delimiter as possible. It might be that it's actually tabs, but that doesn't matter.

We check if the user in the first column is one we care about, by seeing if there is a hash entry for it. We then take the column numbers we care about and sum them up. I have made the assumption that we don't care about the last column because it doesn't look like it's a numeric value.

We need to clean up the units (or whatever they are). We do that by substituting all characters that are not a digit and not a literal dot (.) with a regular expression after assigning it to the variable $number. We then use that to build the sum.

We store this inside our %sums hash, with a key per user we want. Each user has a hash reference with the column numbers as keys. Hashes are not sorted in Perl, so it looks untidy, but that's just the internal representation. They start out as 0 without initialisation.

When we print data structure, we get this output.

$VAR1 = {
          'cp1449' => {
                        '5' => 77572,
                        '1' => 11,
                        '3' => '0.15',
                        '2' => '0.33'
                      },
          'cp1556' => {
                        '2' => '2198.45',
                        '3' => '0.51',
                        '1' => 182,
                        '5' => 115765
                      }
        };

You can retrieve individual values like this:

print $sums{cp1556}{2};

Answer 2

Since you're getting the values in @array, process your array in this way.

You can add the below code in your code once you get @array.

#!/usr/bin/perl

use strict; use warnings;
use Data::Dumper;

my @data = (
'cp1556                                1       0.01re       0.01cp         0avio     22336',
'cp1556                                1       0.01re       0.01cp         0avio     22336k',
'cp1556                               63     862.28re       0.17cp         0avio     13839k',
'cp1556                               43     176.03re       0.13cp         0avio     12083k',
'cp1556                               40     653.49re       0.12cp         0avio     13258k',
'cp1556                               30     506.61re       0.05cp         0avio     12177k',
'cp1556                                4       0.02re       0.02cp         0avio     19736k',
'cp1449                                  6       0.26re       0.11cp         0avio     30176k',
'cp1449                                  3       0.07re       0.04cp         0avio     30261k',
'cp1449                                  2       0.00re       0.00cp         0avio     17135k'
);

my %hash;

foreach my $line (@data){
    my @each_line = split /\s+/, $line;
    $hash{$each_line[0]}{'Col1'} += $each_line[1];
    
    ($each_line[2] = $each_line[2]) =~ s/re$//;
    $hash{$each_line[0]}{'Col2'} += $each_line[2];
    
    ($each_line[3] = $each_line[3]) =~ s/cp$//;
    $hash{$each_line[0]}{'Col3'} += $each_line[3];
    
    ($each_line[4] = $each_line[4]) =~ s/avio$//;
    $hash{$each_line[0]}{'Col4'} += $each_line[4];
    
    ($each_line[5] = $each_line[5]) =~ s/k$//;
    $hash{$each_line[0]}{'Col5'} += $each_line[5];
}

foreach my $cp(keys %hash){
    print "$cp, $hash{$cp}{'Col1'}, $hash{$cp}{'Col2'}re, $hash{$cp}{'Col3'}cp, $hash{$cp}{'Col4'}avio, $hash{$cp}{'Col5'}k\n";
}

Let me know you're getting expected answer here.