CODEWITHSUNDEEP
bashshellawktext-processinggawk
Murlidhar Fichadia
Murlidhar Fichadia

Reputation: 2609

Min-Max Normalization using AWK

I dont know Why I am unable to loop through all the records. currently it goes for last record and prints the normalization for it.

Normalization formula:

New_Value = (value - min[i]) / (max[i] - min[i])

Program

{
    for(i = 1; i <= NF; i++)
    {
        if (min[i]==""){  min[i]=$i;}     #initialise min
        if (max[i]==""){  max[i]=$i;}     #initialise max
        if ($i<min[i]) {  min[i]=$i;}     #new min
        if ($i>max[i]) {  max[i]=$i;}     #new max
    }

}
END {
    for(j = 1; j <= NF; j++)
        {
        normalized_value[j] = ($j - min[j])/(max[j] - min[j]);
        print $j, normalized_value[j];
    }
}

Dataset

4 14 24 34
3 13 23 33 
1 11 21 31
2 12 22 32
5 15 25 35

Current Output

5 1
15 1
25 1
35 1

Required Output

0.75 0.75 0.75 0.75
0.50 0.50 0.50 0.50 
0.00 0.00 0.00 0.00
0.25 0.25 0.25 0.25
1.00 1.00 1.00 1.00

Upvotes: 3

Views: 463

Answers (2)

Mithun B
Mithun B

Reputation: 284

The given answer uses same file to be loaded twice, this can be avoided with following modified script:

# initialization on min, max and value array to be used later
NR == 1 {
    for (i=1; i<=NF; i++) {
        value[i] = $i
        min[i] = $i
        max[i] = $i
    }
}
# finding min and max for each column
NR > 1 {
    for (i=1; i<=NF; i++) {
        value[((NR-1)*NF)+i] = $i
        if      ($i < min[i])    {min[i] = $i}
        else if ($i > max[i])    {max[i] = $i}
    }
}
END {
    nrows = NF
    ncolumns = NR
    for (i=0; i<(ncolumns); i++ ) {
        for (j=1; j<(nrows); j++ ) {
            printf "%.2f%s", (value[(i*nrows)+j]-min[j])/(max[j]-min[j]), OFS
        }
        printf "%.2f\n", (value[(i*nrows)+j]-min[j])/(max[j]-min[j])
    }
}

Save the above awk script as norm.awk. You can run this from shell (and redirect if needed) as:

awk -f norm.awk data.txt > norm_output.txt

or you can run this norm.awk script from vim itself as:

:%!awk -f norm.awk

Which will replace the existing values with min-max normalized values.

Upvotes: 0

glenn jackman
glenn jackman

Reputation: 247012

I would process the file twice, once to determine the minima/maxima, once to calculate the normalized values:

awk '
    NR==1 {
        for (i=1; i<=NF; i++) {
            min[i]=$i
            max[i]=$i
        }
        next
    }
    NR==FNR {
        for (i=1; i<=NF; i++) {
            if      ($i < min[i]) {min[i]=$i}
            else if ($i > max[i]) {max[i]=$i}
        }
        next
    }
    {
        for (i=1; i<=NF; i++) printf "%.2f%s", ($i-min[i])/(max[i]-min[i]), FS
        print ""
    }
' file file
# ^^^^ ^^^^  same file twice!

outputs

0.75 0.75 0.75 0.75 
0.50 0.50 0.50 0.50 
0.00 0.00 0.00 0.00 
0.25 0.25 0.25 0.25 
1.00 1.00 1.00 1.00

Upvotes: 5

Related Questions