CODEWITHSUNDEEP
bashshellunix
mb47
mb47

Reputation: 303

Want to find the distinct lines between two files

I have two files abc.log and xyz.log

abc.log has something like: 

error: 142
error: 143
error: 144
error: 100
error: 142

and file errors.log has content 

error: 140
error: 143
error: 142
error: 144
error: 156

I want to list the error that is not present in abc.log and is present in error.log. abc.log file can have multiple repeating errors.

I am writing a shell script and a one liner would help. Thanks Ambarish

Upvotes: 1

Views: 1704

Answers (3)

Adaephon
Adaephon

Reputation: 18329

This should do the trick:

comm -13 <(sort abc.log | uniq) <(sort errors.log | uniq)

Explanation

  • comm [OPTION]... FILE1 FILE2 compares the (sorted) files FILE1 and FILE2 line by line. Without options it produces a three-column output: lines only in FILE1, lines only in FILE2 and lines in both files. With -1 the first column and with -3 the third column is not shown, so -13 only shows lines unique to FILE2.
  • The construct <(COMMAND) is called process substitution (works at least with bash and zsh). It takes the output of COMMAND and provides a named pipe or file (depending on the capabilities of the system) with the output as content
  • sort FILE | uniq (alternatively sort -u, if your sort supports it) sorts FILE as required by `comm and removes duplicate lines. With duplicates lines, that also are contained in the first file but just appear more often in the second file, would also be shown.

Upvotes: 1

Mahesh Kharvi
Mahesh Kharvi

Reputation: 399

fgrep -v -f abc.log xyz.log

fgrep reads matches from file.

-v prints non matching content

Upvotes: 2

Kent
Kent

Reputation: 195039

awk 'NR==FNR{a[$0];next}!($0 in a)' abc.log errors.log

will do the job, and output:

error: 140
error: 156

idea is save abc.log in a hashtable, for each line in errors.log check if the hashtable a doesn't contain the line, print.

Upvotes: 0

Related Questions