CODEWITHSUNDEEP
pythonsorting
h3tr1ck
h3tr1ck

Reputation: 909

Sorting Problems when using a list

I have a .txt file that contains a list of IP address:

111.67.74.234:8080
111.67.75.89:8080
12.155.183.18:3128
128.208.04.198:2124
142.169.1.233:80

There's a lot more than that though :)

Anyway, imported this into a list using Python and I'm trying to get it to sort them, but I'm having trouble. Anybody have any ideas?

EDIT: Ok since that was vague, this is what I had so fair.

f = open("/Users/jch5324/Python/Proxy/resources/data/list-proxy.txt", 'r+')
lines = [x.split() for x in f]
new_file = (sorted(lines, key=lambda x:x[:18]))

Upvotes: 3

Views: 394

Answers (3)

jamylak
jamylak

Reputation: 133744

@Ninjagecko's solution is the best but here is another way of doing it using re:

>>> import re
>>> with open('ips.txt') as f:
        print sorted(f, key=lambda line: map(int, re.split(r'\.|:', line.strip())))


['12.155.183.18:3128\n', '111.67.74.234:8080\n', '111.67.75.89:8080\n',
'128.208.04.198:2124\n', '142.169.1.233:80 \n']

Upvotes: 0

ninjagecko
ninjagecko

Reputation: 91149

You're probably sorting them by ascii string-comparison ('.' < '5', etc.), when you'd rather that they sort numerically. Try converting them to tuples of ints, then sorting:

def ipPortToTuple(string):
    """
        '12.34.5.678:910' -> (12,34,5,678,910)
    """
    ip,port = string.strip().split(':')
    return tuple(int(i) for i in ip.split('.')) + (port,)

with open('myfile.txt') as f:
    nonemptyLines = (line for line in f if line.strip()!='')
    sorted(nonemptyLines, key=ipPortToTuple)

edit: The ValueError you are getting is because your text files are not entirely in the #.#.#.#:# format as you imply. (There may be comments or blank lines, though in this case the error would hint that there is a line with more than one ':'.) You can use debugging techniques to home in on your issue, by catching the exception and emitting useful debugging data:

def tryParseLines(lines):
    for line in lines:
        try:
            yield ipPortToTuple(line.strip())
        except Exception:
            if __debug__:
                print('line {} did not match #.#.#.#:# format'.format(repr(line)))

with open('myfile.txt') as f:
    sorted(tryParseLines(f))

I was a bit sloppy in the above, in that it still lets some invalid IP addresses through (e.g. #.#.#.#.#, or 257.-1.#.#). Below is a more thorough solution, which allows you do things like compare IP addresses with the < operators, also making sorting work naturally:

#!/usr/bin/python3

import functools
import re

@functools.total_ordering
class Ipv4Port(object):
    regex = re.compile(r'(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3}):(\d{1,5})')

    def __init__(self, ipv4:(int,int,int,int), port:int):
        try:
            assert type(ipv4)==tuple and len(ipv4)==4, 'ipv4 not 4-length tuple'
            assert all(0<=x<256 for x in ipv4), 'ipv4 numbers not in valid range (0<=n<256)'
            assert type(port)==int, 'port must be integer'
        except AssertionError as ex:
            print('Invalid IPv4 input: ipv4={}, port={}'.format(repr(ipv4),repr(port)))
            raise ex

        self.ipv4 = ipv4
        self.port = port

        self._tuple = ipv4+(port,)

    @classmethod
    def fromString(cls, string:'12.34.5.678:910'):
        try:
            a,b,c,d,port = cls.regex.match(string.strip()).groups()
            ip = tuple(int(x) for x in (a,b,c,d))
            return cls(ip, int(port))
        except Exception as ex:
            args = list(ex.args) if ex.args else ['']
            args[0] += "\n...indicating ipv4 string {} doesn't match #.#.#.#:# format\n\n".format(repr(string))
            ex.args = tuple(args)
            raise ex

    def __lt__(self, other):
        return self._tuple < other._tuple
    def __eq__(self, other):
        return self._tuple == other._tuple

    def __repr__(self):
        #return 'Ipv4Port(ipv4={ipv4}, port={port})'.format(**self.__dict__)
        return "Ipv4Port.fromString('{}.{}.{}.{}:{}')".format(*self._tuple)

and then:

def tryParseLines(lines):
    for line in lines:
        line = line.strip()
        if line != '':
            try:
                yield Ipv4Port.fromString(line)
            except AssertionError as ex:
                raise ex
            except Exception as ex:
                if __debug__:
                    print(ex)
                raise ex

Demo:

>>> lines = '222.111.22.44:214 \n222.1.1.1:234\n 23.1.35.6:199'.splitlines()
>>> sorted(tryParseLines(lines))
[Ipv4Port.fromString('23.1.35.6:199'), Ipv4Port.fromString('222.1.1.1:234'), Ipv4Port.fromString('222.111.22.44:214')]

Changing the values to be for example 264... or ...-35... will result in the appropriate errors.

Upvotes: 5

nate_weldon
nate_weldon

Reputation: 2349

You can pre-proces the list so it can be sorted using the built in comparison function. and then process it back to a more normal format.

strings will be the same length and can be sorted . Afterwards, we simply remove all spaces.

you can google around and find other examples of this.

for i in range(len(address)):
    address[i] = "%3s.%3s.%3s.%3s" % tuple(ips[i].split("."))
address.sort()
for i in range(len(address)):
    address[i] = address[i].replace(" ", "")

if you have a ton of ip address you are going to get better processing time if you use c++. it will be more work up front but you will get better processing times.

Upvotes: -1

Related Questions