Python: Generate IP ranges from list of IP addresses

Question

I have a .csv file with list of IP addresses for few data centers. The list currently looks similar to the table below:

  Data_Center_Name      IP
              DC_1      52.102.182.2
              DC_1      52.102.182.4
              DC_1      52.102.182.1
              DC_1      52.102.182.5
              DC_1      52.102.182.3
              DC_1      27.101.178.17
              DC_1      27.101.178.16
              DC_1      27.101.178.15
              DC_1      23.201.165.7
              DC_2      55.200.162.10
              DC_2      55.200.162.12
              DC_2      55.200.162.13
              DC_2      55.200.162.11
              DC_3      30.101.102.4

I want to convert the lists into individual lists such as:

     DC_1 = [52.102.182.1-52.102.182.5,
             27.101.178.15-27.101.178.17,
             23.201.165.7]
     DC_2 = [55.200.162.10-55.200.162.13]
     DC_3 = [30.101.102.4]

Can anyone help me using python?

Answer 1

Using python3 (I can use python2 if requested)

Utilizing the ipaddress and groupby builtin libraries and other builtin goodies:

Create function that converts a list of ipaddress objects to ranges:

def create_range(ip_addresses):
    groups=[]
    for _, g in itertools.groupby(enumerate(sorted(ip_addresses)), lambda (i,x):i-int(x)):
       group = map(operator.itemgetter(1), g)
       if len(group) > 1:
           groups.append("{}-{}".format(group[0], str(group[-1]).split('.')[-1]))
       else:
           groups.append(str(group[0]))
    return groups

import needed libraries, parse out values from csv (using StringIO to mimic reading from file):

import csv ## for reading csv file
import ipaddress ## for creating ip address objects
import io ## for mimicking reading csv file
import operator ## for grouping operation
import itertools ## for grouping operation
import collections ## for creating a defaultdict

ips = defaultdict(list)
csv_file = u"""Data_Center_Name,      IP
              DC_1,      50.102.182.2
              DC_1,      52.102.182.4
              DC_1,      52.102.182.1
              DC_1,      52.102.182.5
              DC_1,      52.102.182.3
              DC_1,      27.101.178.17
              DC_1,      27.101.178.16
              DC_1,      27.101.178.15
              DC_1,      23.201.165.7
              DC_2,      55.200.162.10
              DC_2,      55.200.162.12
              DC_2,      55.200.162.13
              DC_2,      55.200.162.11
              DC_3,      30.101.102.4
"""

with io.StringIO(csv_file) as f:
    reader = list(csv.reader(f))
    for (dc, ip) in reader[1:]:
        ip = ipaddress.IPv4Address(unicode(ip.strip()))
        ips[dc.strip()].append(ip)
    result = {dc: create_range(ip_range) for dc, ip_range in ips.items()}

Result:

In [92]: result
Out[92]:
{'DC_1': ['23.201.165.7',
  '27.101.178.15-17',
  '50.102.182.2',
  '52.102.182.1',
  '52.102.182.3-5'],
 'DC_2': ['55.200.162.10-13'],
 'DC_3': ['30.101.102.4']}

Python2

import csv ## for reading csv file
import ipaddress ## for creating ip address objects
from StringIO import StringIO  ## for mimicking reading csv file
import operator ## for grouping operation
import itertools ## for grouping operation
import collections ## for creating a defaultdict

def create_range(ip_addresses):
    groups=[]
    for _, g in itertools.groupby(enumerate(sorted(ip_addresses)), lambda (i,x):i-int(x)):
       group = map(operator.itemgetter(1), g)
       if len(group) > 1:
           groups.append("{}-{}".format(group[0], str(group[-1]).split('.')[-1]))
       else:
           groups.append(str(group[0]))
    return groups

ips = collections.defaultdict(list)

csv_file = """Data_Center_Name,      IP
              DC_1,      50.102.182.2
              DC_1,      52.102.182.4
              DC_1,      52.102.182.1
              DC_1,      52.102.182.5
              DC_1,      52.102.182.3
              DC_1,      27.101.178.17
              DC_1,      27.101.178.16
              DC_1,      27.101.178.15
              DC_1,      23.201.165.7
              DC_2,      55.200.162.10
              DC_2,      55.200.162.12
              DC_2,      55.200.162.13
              DC_2,      55.200.162.11
              DC_3,      30.101.102.4
"""

reader = csv.reader(StringIO(csv_file))
next(reader)
for (dc, ip) in reader:
    ip = ipaddress.IPv4Address(unicode(ip.strip()))
    ips[dc.strip()].append(ip)
result = {dc: create_range(ip_range) for dc, ip_range in ips.items()}

result with py2 code

print result
{'DC_2': ['55.200.162.10-13'], 'DC_3': ['30.101.102.4'], 'DC_1': ['23.201.165.7', '27.101.178.15-17', '50.102.182.2', '52.102.182.1', '52.102.182.3-5']}

Did works! Thanks. It is possible to get the output: {'DC_2': ['55.200.162.10-55.200.162.13'], 'DC_3': ['30.101.102.4'], 'DC_1': ['23.201.165.7', '27.101.178.15-27.101.178.17', '50.102.182.2', '52.102.182.1', '52.102.182.3-52.102.182.5']} –

Yes, change this line:

groups.append("{}-{}".format(group[0], str(group[-1]).split('.')[-1]))

To this:

groups.append("{}-{}".format(group[0], group[-1]))

Answer 2

My solution is:

1. Convert each IP to decimal number

2. Sort and get ranges (interval) from the list numbers

3. Convert them to IP format.

Input:

ips = [ "52.102.182.2", "52.102.182.4", "52.102.182.1", "52.102.182.5", "52.102.182.3", 
        "27.101.178.17", "27.101.178.16", "27.101.178.15",
        "23.201.165.7", ]

Step 1:

IP => Binary => Decimal

# Convert ips to binary strings
bins = [''.join([bin(int(i))[2:].zfill(8) for i in ip.split('.')]) for ip in ips]

# Convert binary strings to decimal numbers
numbers = [int(b, 2) for b in bins]

or IP => Decimal

# Convert ips to decimal numbers
numbers = [sum((256 ** (3 - k)) * int(n) for k, n in enumerate(ip.split('.'))) for ip in ips]

Step 2:

# Sort decimal numbers
numbers.sort()

# Get ranges from decimal numbers
ranges = []
tmp = []
for i in range(len(numbers)):
    tmp.append(numbers[i])
    if (i == len(numbers) - 1) or (numbers[i + 1] > numbers[i] + 1):
        if len(tmp) == 1:
            ranges.append(tmp[0])
        else:
            ranges.append((tmp[0], tmp[-1]))
        tmp = []

Step 3:

# Convert dec ranges to ip ranges
def dec_to_ip(n):
    return '.'.join([str(int(n % 256 ** (4 - k) / 256 ** (3 - k))) for k in range(4)])

# Final result
ip_ranges = [(dec_to_ip(r[0]), dec_to_ip(r[1])) if type(r) == tuple else dec_to_ip(r) for r in ranges]

Output:

['23.201.165.7', ('27.101.178.15', '27.101.178.17'), ('52.102.182.1', '52.102.182.5')]

Answer 3

*This answer has been edit, cause careless reading the question *

For single list range

df[['P1','P2']]=df.IP.str.rsplit('.',1).apply(pd.Series)
d=df.sort_values(['Data_Center_Name','P1','P2']).\
    groupby(['Data_Center_Name','P1']).\
       IP.apply(lambda x : x.iloc[0]+'-'+x.iloc[-1] if len(x)>1 else x.iloc[0] )
d
Out[388]: 
Data_Center_Name  P1        
DC_1              23.201.165                   23.201.165.7
                  27.101.178    27.101.178.15-27.101.178.17
                  50.102.182                   50.102.182.2
                  52.102.182      52.102.182.1-52.102.182.5
DC_2              55.200.162    55.200.162.10-55.200.162.13
DC_3              30.101.102                   30.101.102.4
Name: IP, dtype: object

For getting you result

d.groupby(level=0).apply(list)
Out[392]: 
Data_Center_Name
DC_1    [23.201.165.7, 27.101.178.15-27.101.178.17, 50...
DC_2                        [55.200.162.10-55.200.162.13]
DC_3                                       [30.101.102.4]
Name: IP, dtype: object