CODEWITHSUNDEEP
bashawkip-addresswhois
user3788019
user3788019

Reputation: 15

Lining up pipeline results alongside input (here, "ip" and whois grep results)

I need to perform a whois lookup on a file containing IP addresses and output both the country code and the IP address into a new file. In my command so far I find the IP addresses and get a unique copy that doesn't match allowed ranges. Then I run a whois lookup to find out who the foreign addresses are. Finally it pulls the country code out. This works great, but I can't get it show me the IP alongside the country code since that isn't included in the whois output.

What would be the best way to include the IP address in the output?

awk '{match($0,/[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/); ip = substr($0,RSTART,RLENGTH); print ip}' myInputFile \
  | sort \
  | uniq \
  | grep -v '66.33\|66.128\|75.102\|216.106\|66.6' \
  | awk -F: '{ print "whois " $1 }' \
  | bash \
  | grep 'country:' \
  >> myOutputFile

I had thought about using tee, but am having troubles lining up the data in a way that makes sense. The output file should be have both the IP Address and the country code. It doesn't matter if they are a single or double column.

Here is some sample input:

Dec 27 04:03:30 smtpfive sendmail[14851]: tBRA3HAx014842: to=, delay=00:00:12, xdelay=00:00:01, mailer=esmtp, pri=1681345, relay=redcondor.itctel.c om. [75.102.160.236], dsn=4.3.0, stat=Deferred: 451 Recipient limit exceeded for this se nder Dec 27 04:03:30 smtpfive sendmail[14851]: tBRA3HAx014842: to=, delay=00:00:12, xdelay=00:00:01, mailer=esmtp, pri=1681345, relay=redcondor.itctel.c om. [75.102.160.236], dsn=4.3.0, stat=Deferred: 451 Recipient limit exceeded for this se nder

Thanks.

Upvotes: 0

Views: 209

Answers (1)

Charles Duffy
Charles Duffy

Reputation: 295914

In general: Iterate over your inputs as shell variables; this then lets you print them alongside each output from the shell.

The below will work with bash 4.0 or newer (requires associative arrays):

#!/bin/bash
#      ^^^^- must not be /bin/sh, since this uses bash-only features

# read things that look vaguely like IP addresses into associative array keys
declare -A addrs=( )
while IFS= read -r ip; do
  case $ip in 66.33.*|66.128.*|75.102.*|216.106.*|66.6.*) continue;; esac
  addrs[$ip]=1
done < <(grep -E -o '[0-9]+[.][0-9]+[.][0-9]+[.][0-9]+')

# getting country code from whois for each, printing after the ip itself
for ip in "${!addrs[@]}"; do
  country_line=$(whois "$ip" | grep -i 'country:')
  printf '%s\n' "$ip $country_line"
done

An alternate version which will work with older (3.x) releases of bash, using sort -u to generate unique values rather than doing that internal to the shell:

while read -r ip; do
  case $ip in 66.33.*|66.128.*|75.102.*|216.106.*|66.6.*) continue;; esac
  printf '%s\n' "$ip $(whois "$ip" | grep -i 'country:')"
done < <(grep -E -o '[0-9]+[.][0-9]+[.][0-9]+[.][0-9]+' | sort -u)

It's more efficient to perform input and output redirection for the script as a whole than to put a >> redirection after the printf itself (which would open the file before each print operation and close it again after, incurring a substantial performance penalty), which is why suggested invocation for this script looks something like:

countries_for_addresses </path/to/logfile >/path/to/output

Upvotes: 2

Related Questions