CODEWITHSUNDEEP
bashawk
user53029
user53029

Reputation: 695

print last occurrence of each unique line by IP in file

I need to parse a log file so that the following entries like this:

Jul 23 17:38:06 192.168.1.100 638 "this message will always be the same"
Jul 23 17:56:11 192.168.1.100 648 "this message will always be the same."


Jul 23 18:14:17 192.168.1.101 "this message will always be the same."
Jul 23 18:58:17 192.168.1.101 "this message will always be the same."

Look like this:

Jul 23 17:56:11 192.168.1.100 648 "this message will always be the same."
Jul 23 18:58:17 192.168.1.101 "this message will always be the same."

Basically what I am doing is taking a file that has duplicate IP addresses but with different timestamps, and finding the last occurrence (or most recent by time) of each IP address, and printing that to the screen or directing it into another file.

What I have tried:

I have written a bash script that I thought would allow me to do this but it is not working. 

#!/bin/bash

/bin/grep 'common pattern to all lines' /var/log/file | awk '{print $4}' | sort - u > /home/user/iplist

while IFS='' read -r line || [[ -n "$line" ]]; do
echo "$line"
done < "/home/user/iplist"

awk '/'$line'/ {a=$0}END{print a} ' /var/log/logfile

The script runs and outputs each IP address, but it does not print the whole line except for the last one.

ex..

 192.168.100.101
 192.168.100.102
 192.168.100.103
 Jul 23 20:20:55 192.168.100.104 "this message will always be the same."

The first command in the script takes all unique occurrences of an IP and sends that to a file. The while loop assigns a "$line" variable to each line which is then passed to awk which I thought would take each IP then search the actual file and print out the last occurrance of each one. How can I get this to work, either with a script or perhaps an awk one liner?

Upvotes: 0

Views: 360

Answers (2)

anubhava
anubhava

Reputation: 785088

You can use this awk command:

awk 'NF{a[$4]=$0} NF && !seen[$4]++{ips[++numIps]=$4} END {
  for (i=1;i<=numIps;i++) print a[ips[i]] }' file

Jul 23 17:56:11 192.168.1.100 648 "this message will always be the same."
Jul 23 18:58:17 192.168.1.101 "this message will always be the same."

Upvotes: 1

Ed Morton
Ed Morton

Reputation: 203364

$ tac file | awk '!seen[$4]++' | tac
Jul 23 17:56:11 192.168.1.100 648 "this message will always be the same."
Jul 23 18:58:17 192.168.1.101 "this message will always be the same."

Upvotes: 5

Related Questions