CODEWITHSUNDEEP
bashshellgrep
stranksa
stranksa

Reputation: 33

Checking if strings in one file occur in another set of files, list those that don't

I have a situation where I have a large number of files that I need to check if they contain a string listed, one per line, in another file and to report where they do not. The string matching file in a list of VLANs and the large number of files to be checked are the period outputs of 'show mac-address' from our Core Switches, these being a bunch of txt files. I am using a Linux Bash shell.

I can cover of the matching using grep easy enough with...

cat *.txt > [MAC-File] && fgrep -of [VLAN-File] [MAC-File] | sort -h | uniq -c

Which gives me a list of the VLANs that match and the number of lines in the txt files the do. That's progress but what I need is to find the VLANs that don't have MAC addresses seen in them so I need to turn the logic around. My searching tells me grep doesn't have an opposite condition to the -o so I need to find an alternative. This is to be applied against 3 very large LANs each with hundreds of VLANs in them, and I don't want to input the results into an Excel spreadsheet!

Please note the files I am checking against have more data per line that just the VLAN number so compare lines does not work.

The first file with the strings to be looked for (or not for!) is in the format..

 100
 103 
 230

Note I have space before and after each number to make them unique so they only match the second column of the large data file I am checking which is in the format

6c4b-904b-0c5c   230        Learned          BAGG103                  Y

Upvotes: 3

Views: 1274

Answers (1)

kvantour
kvantour

Reputation: 26481

Since you make use of fgrep, which is synonymous to grep -F we know that the pattern file are fixed strings. To find which patters did not match, you use the following method:

$ grep -oFf pattern_file search_file | grep -voFf - pattern_file

In case of the OP, this becomes:

$ grep -oFf [VLAN-File] [MAC-File] | grep -voFf - [VLAN-File]

You can also do this with awk in a single go:

$ awk '(NR==FNR){a[$0];next}($2 in a){a[$2]++}END{for(i in a) if (a[i]==0) print i}' [VLAN-File] [MAC-File]

The above works for exact matches, so no need to have the extra spaces. If you want to keep the extra spaces, it is a bit more tricky:

$ awk '(NR==FNR){a[$0];next}
       {for(i in a) if (i ~ $0) a[i]++}
       END{for(i in a) if (a[i]==0) print i}' [VLAN-File] [MAC-File]

All the above will print the VLAN-File entries that do not appear in the MAC-File

Upvotes: 3

Related Questions