CODEWITHSUNDEEP
pythonpandas
Katarina Alves
Katarina Alves

Reputation: 89

How to take specific ip addresses from log file in shell script and convert to Python

I am new to python and I have a log file, which consists hundreds of destination ip, and I have taken just the destination ip addresses from log file and this is the code

f = open('/Users/kiya/Desktop/webgw logs/check/1st.log', 'r')
list_ip = sorted([])
for line in f:
     k = line.split()[3]
     list_ip.append(k)
ips = list(set(list_ip))
for ip in ips:
    #print(ip,'Count',list_ip.count(ip))
    print (ip)

And how can I filter the IP address based on the shell script below in python

cat 1st.log | grep -e "proto=6" -e "proto=17" | grep -e "srcip=10\." -e "srcip=172\." -e "srcip=192\.168" | grep -v -e "dstip=10\." -e "dstip=169\.254" -e "dstip=17\." -e "dstip=208\.91\.11[2-3]\." -e "dstip=172\." | cut -d ' ' -f 7 | sort | uniq | awk 'match($0,/[0-9]+.[0-9]+.[0-9]+.[0-9]+/){print substr($0, RSTART, RLENGTH)}' > list.txt

This is my basic log file, and there are many data in it

#time_stamp "auth_user" dest_ip src_ip status_code "req_line" "categories" "rep_level" "media_type" bytes_to_client bytes_from_client bytes_to_server bytes_from_server "user_agent" "virus_name" block_res "application_name"
[30/Dec/2017:23:59:47 +0900] "" 255.255.255.255 172.16.23.177 407 "CONNECT dapi.ds-intel.net:443 HTTP/1.1" "" "-" "" 2935 243 0 0 "Devicescape-Agent/2.0.999 (Windows 7 - 34525574-f744-a01f-f5a2-bbca6285294e) tmobile_wispr1" "" 81 ""
[30/Dec/2017:23:59:47 +0900] "" 255.255.255.255 172.16.23.177 407 "CONNECT dapi.ds-intel.net:443 HTTP/1.1" "" "-" "" 2946 327 0 0 "Devicescape-Agent/2.0.999 (Windows 7 - 34525574-f744-a01f-f5a2-bbca6285294e) tmobile_wispr1" "" 81 "

Upvotes: 0

Views: 180

Answers (1)

MaxU - stand with Ukraine
MaxU - stand with Ukraine

Reputation: 210842

IIUC:

import pandas as pd

ips = pd.read_csv('/Users/kiya/Desktop/webgw logs/check/1st.log',
                  delim_whitespace=True, usecols=[3], squeeze=True)
print(ips.unique())

Upvotes: 1

Related Questions