CODEWITHSUNDEEP
linuxshellawk
rakesh kandukuri
rakesh kandukuri

Reputation: 305

How to ignore specific column when comparing two files using awk

File1:

1|footbal|play1
2|cricket|play2
3|tennis|play3
5|golf|play5

File2:

1|footbal|play1
2|cricket|play2
3|tennis1|play3
4|soccer|play4
5|golf|play6

Output File:

4|soccer|play4
5|golf|play6

I am comparing all columns of file1 and file2, but I need to ignore the second column while comparing.

awk 'NR==FNR {exclude[$0];next} !($0 in exclude)' file1 file2 > file3

Upvotes: 0

Views: 1436

Answers (3)

RavinderSingh13
RavinderSingh13

Reputation: 133438

EDIT2(Generic solution): To nullify more than 1 column(s) in both the Input_file(s) could try following. I have made variables for it so you need not to hard code fields which you want to nullify in your code. One should mention all field numbers separated with , in -v file1_ignore and file2_ignore variables of this awk program.

awk -v file1_ignore="2,3" -v file2_ignore="2,3" '
BEGIN{
  FS=OFS="|"
  num1=split(file1_ignore,array1,",")
  num2=split(file2_ignore,array2,",")
}
FNR==NR{
  for(i=1;i<=num1;i++){
    $array1[i]=""
  }
  a[$0]
  next
}
{
  val=$0
  for(i=1;i<=num2;i++){
    $array2[i]=""
  }
}
!($0 in a){
  print val
  val=""
}
' file1 file2

Explanation: Adding a detailed explanation for above code.

awk -v file1_ignore="2,3" -v file2_ignore="2,3" '    ##Starting awk program from here and setting variables named file1_ignore(which will be used to ignoring fields in Input_file1), file2_ignore(which will be used to ignoring fields in Input_file2).
BEGIN{                                               ##Starting BEGIN section from here.
  FS=OFS="|"                                         ##Setting field seaprator and output field separator as | here.
  num1=split(file1_ignore,array1,",")                ##Spitting file1_ignore variable to array1 here with separator as , here.
  num2=split(file2_ignore,array2,",")                ##Spitting file1_ignore variable to array2 here with separator as , here.
}                                                    ##Closing BEGIN BLOCK for this code here.
FNR==NR{                                             ##Checking condition which will be TRUE for first Input_file Input_file1 here.
  for(i=1;i<=num1;i++){                              ##Starting for loop to run till variable num1 here.
    $array1[i]=""                                    ##Nullifying field(which will be get by value of array1).
  }                                                  ##Closing above for loop BLOCK here.
  a[$0]                                              ##Creating an array with index of current line.
  next                                               ##next will skip all further statements from here.
}                                                    ##Closing BLOCK for FNR==NR condition here.
{
  val=$0                                             ##Creating a variable val whose value is current line.
  for(i=1;i<=num2;i++){                              ##Starting for loop to run till variable num2 here.
    $array2[i]=""                                    ##Nullifying field(which will be get by value of array2).
  }                                                  ##Closing above for loop BLOCK here.
}
!($0 in a){                                          ##Checking condition if current line is NOT present in array a then run futher statements.
  print val                                          ##Printing variable val here.
  val=""                                             ##Nullify variable val here.
}
' file1 file2                                        ##Mentioning Input_file(s) name here.

EDIT1: To ignore multiple and different columns in both the files try following, I have taken example of same column number 2 to be nullified in both files you could keep it as per your need too.

awk -v file1_ignore="2" -v file2_ignore="2" '
BEGIN{
  FS=OFS="|"
}
FNR==NR{
  $file1_ignore=""
  a[$0]
  next
}
{
  val=$0
  $file2_ignore=""
}
!($0 in a){
  print val
  val=""
}
' file1 file2

Could you please try following.

awk 'BEGIN{FS="|"}FNR==NR{a[$1,$3];next} !(($1,$3) in a)' file1 file2

Upvotes: 2

James Brown
James Brown

Reputation: 37394

To ignore:

$ awk -F\| '
NR==FNR {           # first file
    $2=""           # empty the unwanted fields, chain them: $2=$3=...$n=""
    a[$0]           # hash on $0
    next            # next record
}
{                   # second file
    b=$0            # backup the record to b
    $2=""           # empty the same field
    if(!($0 in a))  # refer
        print b     # output the backup
}' file file2

Output:

4|soccer|play4
5|golf|play6

This of course makes sense only if NF >> amount of fields to null. In other case use the other solutions.

Upvotes: 4

jas
jas

Reputation: 10865

The comma , operator is handy here to treat the fields individually. If you just want the first and third fields you can use the same pattern you already have:

$ awk -F\| 'NR==FNR {exclude[$1,$3];next} !(($1,$3) in exclude)' file1 file2
4|soccer|play4
5|golf|play6

Upvotes: 2

Related Questions