CODEWITHSUNDEEP
bashtext-processingnetstat
Florian Feldhaus
Florian Feldhaus

Reputation: 5932

How to append string to file if it is not included in the file?

Protagonists

Setting

The Cron Daemon is repeatedly performing the same job where he forces an innocent netstat to show the network status (netstat -n). Pipes then have to pick up the information and deliver it to bystanding text processing utilities (| grep tcp | awk '{ print $5 }' | cut -d "." -f-4). >> has to scribe the important results to a file. As his highness, The Admin, is a lazy and easily annoyed ruler, >> only wants to scribe new information to the file. 

*/1 * * * * netstat -n | grep tcp | awk '{ print $5 }' | cut -d "." -f-4 >> /tmp/file

Soliloquy by >>

To append, or not append, that is the question: 
Whether 'tis new information to bother The Admin with 
and earn an outrageous Fortune,
Or to take Arms against `netstat` and the others, 
And by opposing, ignore them? To die: to sleep;

note by the publisher: For all those that had problems understanding Hamlet, like I did, the question is, how do I check if the string is already included in the file and if not, append it to the file?

Upvotes: 3

Views: 449

Answers (2)

Kai Sternad
Kai Sternad

Reputation: 22830

What a piece of work is piping, how easy to reason about, 
how infinite in use cases, in bash and script, 
how elegant and admirable in action,
how like a vim in flexibility, 
how like a gnu!

Here is a slightly different take:

netstat -n | awk -F"[\t .]+" '/tcp/ {print $9"."$10"."$11"."$12}' | sort -nu | while read ip; do if ! grep -q $ip /tmp/file; then echo $ip >> /tmp/file; fi; done;

Explanation:

awk -F"[\t .]+" '/tcp/ {print $9"."$10"."$11"."$12}'

Awk splits the input string by tabs and ".". The input string is filtered (instead of using a separate grep invocation) by lines containing "tcp". Finally the resulting output fields are concatenated with dots and printed out.

sort -nu

Sorts the IPs numerically and creates a set of unique entries. This eliminates the need for the separate uniq command.

if ! grep -q $ip /tmp/file; then echo $ip >> /tmp/file; fi;

Greps for the ip in the file, if it doesn't find it, the ip gets appended.

Note: This solution does not remove old entries and clean up the file after each run - it merely appends - as your question implied.

Upvotes: 1

urish
urish

Reputation: 9033

Unless you are dealing with a very big file, you can use the uniq command to remove the duplicate lines from the file. This means you will also have the file sorted, I don't know if this is an advantage or disadvantage for you:

netstat -n | grep tcp | awk '{ print $5 }' | cut -d "." -f-4 >> /tmp/file && sort /tmp/file | uniq > /tmp/file.uniq

This will give you the sorted results without duplicates in /tmp/file.uniq

Upvotes: 1

Related Questions