CODEWITHSUNDEEP
pythonpandasnumpydataframenetwork-programming
sectechguy
sectechguy

Reputation: 2117

Pandas using netaddr on 2 dataframes to see if ip column falls in cidr column with boolean outcome

I am working with the netaddr python library. I have 2 dataframes one with IP ranges that I convert to CIDR notation and one that has IP address that I would like to see if they fall in any of the ranges.

Create Range Dataframe:

import pandas as pd
import netaddr
from netaddr import *

a = {'StartAddress': ['65.14.88.64', '148.77.37.88', '65.14.41.128', '65.14.40.0'],
 'EndAddress': ['65.14.88.95', '148.77.37.95','65.14.41.135', '65.14.40.255']}
df1 = pd.DataFrame(data=a)

#Convert range to netaddr cidr format
def rangetocidr(row):
    return netaddr.iprange_to_cidrs(row.StartAddress, row.EndAddress)    

df1["CIDR"] = df1.apply(rangetocidr, axis=1)

df1
    StartAddress  EndAddress    CIDR
0   65.14.88.64   65.14.88.95   [65.14.88.64/27]
1   148.77.37.88  148.77.37.95  [148.77.37.88/29]
2   65.14.41.128  65.14.41.135  [65.14.41.128/29]
3   65.14.40.0    65.14.40.255  [65.14.40.0/24]

df1["CIDR"].iloc[0]
[IPNetwork('65.14.88.64/27')]

Create IP dataframe:

b = {'IP': ['65.13.88.64', '148.65.37.88','65.14.88.65','148.77.37.93','66.15.41.132']}
df2 = pd.DataFrame(data=b)

#Convert ip to netaddr format
def iptonetaddrformat (row):
    return netaddr.IPAddress(row.IP)

df2["IP_Format"] = df2.apply(iptonetaddrformat, axis=1)
df2
    IP            IP_Format
0   65.13.88.64   65.13.88.64
1   148.65.37.88  148.65.37.88
2   65.14.88.65   65.14.88.65
3   148.77.37.93  148.77.37.93
4   66.15.41.132  66.15.41.132

df2["IP_Format"].iloc[0]
IPAddress('65.13.88.64')

I am looking to add a column to df2 if the ips are in the cidr blocks from df1. So it would look like:

df2
    IP            IP_Format     IN_CIDR
0   65.13.88.64   65.13.88.64   False
1   148.65.37.88  148.65.37.88  False
2   65.14.88.65   65.14.88.65   True
3   148.77.37.93  148.77.37.93  True
4   66.15.41.132  66.15.41.132  False

I would prefer to perform this just using columns from 2 dataframes but have tried this by converting the columns to lists and using the following, but this doesnt seem to work:

df2list = repr(df2[['IP_Format']])
df1list = df[['CIDR']]

def ipincidr (row):
    return netaddr.largest_matching_cidr(df2list, df1list)

df2['INRANGE'] = df2.apply(ipincidr, axis=1)

Upvotes: 1

Views: 450

Answers (1)

meW
meW

Reputation: 3967

The following solution is based on the assumption that only fourth group of IP changes and first three remains intact as shown in question.

# Splitting IP into 2 parts __.__.__ and __. 
# Doing this for IP from df2 along with Start and End columns from df1

ip = pd.DataFrame(df2.IP.str.rsplit('.', 1, expand=True))
ip.columns = ['IP_init', 'IP_last']

start = pd.DataFrame(df1.StartAddress.str.rsplit('.', 1, expand=True))
start.columns = ['start_init', 'start_last']

end = pd.DataFrame(df1.EndAddress.str.rsplit('.', 1, expand=True))
end.columns = ['end_init', 'end_last']

df = pd.concat([ip, start, end], axis=1)

# Checking if any IP belongs to any of the given blocks, if yes, note their index

index = []
for idx, val in enumerate(df.itertuples()):
    for i in range(df.start_init.count()):
        if df.loc[idx, 'IP_init'] == df.loc[i, 'start_init']:            
            if df.loc[idx, 'IP_last'] >= df.loc[i, 'start_last'] 
                  and df.loc[idx, 'IP_last'] <= df.loc[i, 'end_last']:
                index.append(idx)
                break


# Creating column IN_CIDR and marking True against the row which exists in IP block

df2['IN_CIDR'] = False
df2.loc[index, 'IN_CIDR'] = True
df2

    IP            IP_Format     IN_CIDR
0   65.13.88.64   65.13.88.64   False
1   148.65.37.88  148.65.37.88  False
2   65.14.88.65   65.14.88.65   True
3   148.77.37.93  148.77.37.93  True
4   66.15.41.132  66.15.41.132  False

Note - You may also use np.where to skip first iteration using np.where(df.IP_init.isin(df.start_init), True, False) which results to [False, False, True, True, False] and thus you can focus later only on the True rows and thus reducing overhead.

Upvotes: 1

Related Questions