CODEWITHSUNDEEP
pythonlistsplit
Paul
Paul

Reputation: 5974

Split string into list, then split list into list of lists

I have a string, a sample of it looks like this:

http://pastebin.com/2FFNa3Qx

There are patterns, generally after "Sequence:" the next 22 elements repeat, these are the data that I want isolated.

So I was thinking, if I split the string at Sequence: into a list of elements and then split this generated list into a list of lists via \n, each of these lists that had a length of 22 elements would be the data I wanted. So I've tried that with this code:

    proc = subprocess.Popen(cmd_rancli, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
    //proc_stdout is the string
    proc_stdout  = proc.communicate(ran_opt_get_access_data)[0]

    parse = proc_stdout.split('Sequence:')
    print parse
    time.sleep(5)
    parse2 = [i.split('\n')[0] for i in parse] 
    print parse2
    time.sleep(5)

However the second of these doesn't give me what I expect, what am i doing wrong?

Actual output:

    parse2 = ['RAN> get ap 108352 attr=4192', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' 
', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ']

Splitting and returning a space?

here is some of the result of the first parse: https://i.sstatic.net/sOQHP.png

Upvotes: 0

Views: 1930

Answers (2)

Francis Avila
Francis Avila

Reputation: 31621

The reason you don't get what you expect is that there is whitespace between each Sequence: and the following newline. [i.split('\n')[0] for i in parse] will get the first item after splitting on newlines, which will be that whitespace.

Instead of fixing this approach, I suggest you do something slightly more sophisticated and create a dict that models your output:

def add_data(key, value, data):
    if key.startswith('Value('):
        if key.endswith('(int)'):
            value = int(value)
        data['Sequences'][-1].append(value)
    elif key == 'Sequence':
        data['Sequences'].append([])
    else:
        data[key] = value

def parse_lines(lineseq):
    data = {'Sequences':[]}
    for line in lineseq:
        try:
            key, value = [part.strip() for part in line.split(':', 1)]
        except ValueError:
            continue
        add_data(key, value, data)
    return data

lines = proc_stdout.split('\n')
data = parse_lines(lines)

This produces a data structure like so:

{'AttributeId': '4192',
 'AttributeList': '',
 'ClassId': '1014 (AP)',
 'InstanceId': '0',
 'MessageType': '81 (GetAttributesResponse)',
 'ObjectInstance': '',
 'Protocol': 'BSMIS Rx',
 'RDN': '',
 'TransactionId': '66',
 'Sequences': [[],
               [1,'2013-02-26T15:01:11Z'],
               [],
               [10564,13,388,0,-321,83,'272','05',67,67,708,896,31,128,-12,-109,0,-20,-111,-1,-1,0],
               [10564,13,108,0,-11,83,'272','05',67,67,708,1796,31,128,-12,-109,0,-20,-111,-1,-1,0],
               [10589,16,388,0,-15,79,'272','05',67,67,708,8680,31,125,-16,-110,0,-20,-111,-1,-1,0],
               [10589,15,108,0,-16,81,'272','05',67,67,708,8105,31,126,-14,-109,0,-20,-111,-1,-1,0],
               [10637,40,233,0,-11,89,'272','03',30052,1,5,54013,33,103,-6,-76,1,-20,-111,-1,-1,0],
               [10662,46,234,0,-15,85,'272','03',30052,1,5,54016,33,97,-10,-74,1,-20,-111,-1,-1,0],
               [10712,51,12,0,-24,91,'272','01',4013,254,200,2973,3,62,-4,-63,0,-20,-111,-1,-1,0],
               [10737,15,224,0,-16,82,'272','01',3020,21,21,40770,33,128,-13,-108,0,-20,-111,-1,-1,0],
               [10762,14,450,0,-7,78,'272','01',3020,21,21,53215,29,125,-17,-113,0,-20,-111,-1,-1,0],
               [10762,15,224,0,-7,85,'272','01',3020,21,21,50770,33,128,-10,-105,0,-20,-111,-1,-1,0],
               [10762,14,124,0,-7,78,'272','01',3020,10,10,56880,32,128,-17,-113,0,-20,-111,-1,-1,0],
               [10812,11,135,0,-14,81,'272','02',36002,1,11,43159,31,130,-14,-113,1,-20,-111,-1,-1,0],
               [10837,42,23,0,-9,89,'272','02',36002,1,11,53529,31,99,-6,-74,1,-20,-111,-1,-1,0,54],
               [13,'2013-02-26T15:02:09Z'],
               [],
               [2,12,7,0,9,70,'272','02',20003,0,0,15535,0,0,0,0,1,100,100,-1,-1,0],
               [5,15,44,0,-205,77,'272','02',20003,0,0,15632,0,0,0,0,1,100,100,-1,-1,0],
               [7,25,9,0,0,84,'272','02',20002,0,0,50883,0,0,0,0,1,100,100,-1,-1,0]]
}

If you then only want sequences with a length of 22, it's easy to get that:

len22seqs = [s for s in data['Sequences'] if len(s)==22]
# alternatively:
len22seqs = filter(lambda s: len(s)==22, data['Sequences'])

Upvotes: 1

Vyktor
Vyktor

Reputation: 20997

Using string you provided on pastebin (as content of variable a):

>>> result = [i.strip().split('\n') for i in a.split('Sequence')]
>>> [len(i) for i in result]
[10, 1, 3, 1, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 24, 3, 1, 23, 23, 23]

For example here:

       Sequence:
     Value(int):     1
  Value(string):     2013-02-26T15:01:11Z
       Sequence:

So let's filter only those with 23 elements (note the first element is :):

>>> result = [i[1:] for i in result if len(i) == 23]
>>> [len(i) for i in result]
[22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22]

And now you have array which looks like this:

>>> print( '\n'.join(result[0]))
     Value(int):       10564
     Value(int):       13
     Value(int):       388
     Value(int):       0
     Value(int):       -321
     Value(int):       83
  Value(string):       272
  Value(string):       05
     Value(int):       67
     Value(int):       67
     Value(int):       708
     Value(int):       896
     Value(int):       31
     Value(int):       128
     Value(int):       -12
     Value(int):       -109
     Value(int):       0
     Value(int):       -20
     Value(int):       -111
     Value(int):       -1
     Value(int):       -1
     Value(int):       0

So whole code you need with data you provided is:

proc_stdout = proc.communicate(ran_opt_get_access_data)[0].decode('utf-8')
result = [i.strip().split('\n') for i in proc_stdout.split('Sequence')]
result = [i[1:] for i in result if len(i) == 23]
# Or at least [i[1:] for i in result if len(i) > 1]

Edit - removing value

We'll use simple hack, thus there's always only one : and string.find() and string.strip() to remove white spaces:

def filter_value(text):
    index = text.find( ':')
    # Not found :
    if index < 0:
        return text.strip()

    return text[index+1:].strip()

And implement it by replacing this line:

result = [i[1:] for i in result if len(i) == 23]

With this one-liner:

result = [[filter_value(j) for j in i[1:]] for i in result if len(i) == 23]

Upvotes: 2

Related Questions