CODEWITHSUNDEEP
arraysregexbashmultidimensional-array
e-info128
e-info128

Reputation: 4072

Count ip repeat in log from bash

bash as I can tell from the repetition of an IP within a log through a specific search?

By example:

#!/bin/bash

# Log line: [Sat Jul 04 21:55:35 2015] [error] [client 192.168.1.39] Access denied with status code 403.

grep "status\scode\s403" /var/log/httpd/custom_error_log | while read line ; do

    pattern='^\[.*?\]\s\[error\]\s\[client\s(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\].*?403'
    [[ $line =~ $pattern ]]

    res_remote_addr="${BASH_REMATCH[1]}.${BASH_REMATCH[2]}.${BASH_REMATCH[3]}.${BASH_REMATCH[4]}"

    echo "Remote Addr: $res_remote_addr"

done

I need to know the end results obtained a few times each message 403 ip, if possible sort highest to lowest.

By example output:

200.200.200.200 50 times.
200.200.200.201 40 times.
200.200.200.202 30 times.
... etc ...

This we need to create an html report from a monthly log of apache in a series of events (something like awstats).

Upvotes: 0

Views: 96

Answers (2)

Jason Hu
Jason Hu

Reputation: 6333

there are better ways. following is my proposal, which should be more readable and easier to maintain:

grep -P -o '\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}' log_file | sort | uniq -c | sort -k1,1 -r -n

output should be in a form of:

count1 ip1
count2 ip2

update:

filter only 403:

grep -P -o '\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}(?=.*403)' log_file | sort | uniq -c | sort -k1,1 -r -n

notice that a look ahead would suffice.

Upvotes: 2

Marki555
Marki555

Reputation: 6860

If log file is in the format as mentioned in question, the best is to use awk to filter out the status code needed plus output only the IP. Then use the uniq command to count each occurence:

awk '/code 403/ {print $8}' error.log | sort | uniq -c |sort -n

In awk, we filter by regexp /code 403/ and then for matching lines we print the 8th value (values are separated by whitespace), which is the IP.

Then we need to sort the output, so that the same IPs are one after another - this is requirement of the uniq program.

uniq -c prints each unique line from input only once - and preceded by the number of occurences. Finnaly we sort this list numericaly to get the IPs sorted by count.

Sample output (first is no. of occurences, second is IP):

1 1.1.1.1
10 2.2.2.2
12 3.3.3.3

Upvotes: 0

Related Questions