CODEWITHSUNDEEP
awkgrep
Brian-reins
Brian-reins

Reputation: 31

Extract IP address after pattern is found in log file

I need to extract and then block a certain IP address (IPv4 only) within a log file using ipset command within a bash script. The logfile structure looks like this:

[Line 1] random text
[Line 2] random text
...
[Line 26] random text
Aug 31 13:40:14 [logs ] con_accept: [759] connection from 127.0.0.1:56860 is successfull on 0.0.0.0:25000
[Line 28] random text
...
[Line 38] random text
Aug 31 13:40:57 [logs ] _authrequest: [759] random text client connected and the sky is blue today more random text
[Line 40] random text

So far, I'm either for grep or awk usage. Checking the last 15 lines after pattern has been found is easy, however, I'm trying to do extra checks and extract the correct IP address once the string is found and clientID match.

awk 'c-->0;$0~s{if(b)for(c=b+1;c>1;c--)print r[(NR-c+1)%b];print;c=a}b{r[NR%b]=$0}' b=15 s="sky is blue" logfile

grep -B 15 'sky is blue' logfile

Upvotes: 2

Views: 476

Answers (2)

Ed Morton
Ed Morton

Reputation: 203358

You never additionally need grep (or sed) when you're using awk.

If you have tac on your system then:

$ cat tst.sh
#!/usr/bin/env bash

ip=$(
    tac "${@:--}" |
    awk -v b=15 -v s='sky is blue' '
        $0 ~ s {
            endNr = NR + b
            id = $2
        }
        ($6 == "con_accept:") && ($7 == id) {
            sub(/:.*/,"",$5)
            ip = $10
        }
        NR == endNr { exit }
        END {
            if ( ip == "" ) {
                exit 1
            }
            else {
                print ip
            }
        }
    '
) &&
echo ipset add blacklist "$ip"


$ ./tst.sh file
ipset add blacklist 127.0.0.1

otherwise less efficiently using GNU awk for arrays of arrays:

$ cat tst.sh
#!/usr/bin/env bash

ip=$(
    awk -v b=15 -v s='sky is blue' '
        { id = $7 }
        $6 == "con_accept:" {
            sub(/:.*/,"",$10)
            ips[NR][id] = $10
        }
        $0 ~ s {
            for (n=NR-1; n>(NR-b); n--) {
                if ( (n in ips) && (id in ips[n]) ) {
                    ip = ips[n][id]
                    exit
                }
            }
        }
        END {
            if ( ip == "" ) {
                exit 1
            }
            else {
                print ip
            }
        }
    ' "${@:--}"
) &&
echo ipset add blacklist "$ip"


$ ./tst.sh file
ipset add blacklist 127.0.0.1

Remove the echo when you're done testing and happy with the results.

The above assumes that the [Line N] strings at the start of each line in your question aren't really present. If they are it's an easy tweak to add 2 to each field number use.

Upvotes: 4

markp-fuso
markp-fuso

Reputation: 34244

A simple first pass:

grep -nB 15 'sky is blue' logfile | sort -nr

The grep -n will preface each line with its line number from the file (format: <line_number>:<contents_of_line>).

The sort -rn will sort the rows in reverse order using numeric sorting.

From here you can pass these lines to a bash/while loop or awk script, keeping in mind several (potential) issues:

  • what if 'sky is blue' is not found?
  • what if 'sky is blue' has multiple matches? process the first, the last or all instances?
  • what if a given clientID has multiple matching rows but with different ip addresses? process the first, the last or all ip addresses?
  • do all lines of interest have the same exact formats as displayed in the sample data or could there be different formats that need to be coded for?

'course, all of this could be coded in a dozen different ways via a single awk script (eg, Ed Morton's answer) ...

Upvotes: 1

Related Questions