samtal
samtal

Reputation: 129

Using Python Lists and arrays

After years of working with multidimensional arrays, the Python array concept of 'List' seems quite complex to me (although it is said to be superior).
I have long (thousands of lines) 2-dimensional array files with hexadecimal values with the following format (can be with or without line separation):

[0xF26,0x70,0x66],[0x1158,0x72,0xA6],[0x1388,0x72,0xB6],
[0x15BA,0x4E,0x08],[0x17E9,0x70,0x92],[0x1A1D,0x72,0x94],
[0x1C4F,0x72,0xB4],[0x4409,0x4A,0x14], etc. etc.

I wish to use the file in Python, extracting and manipulating any random element.
I realize that I will have to convert the file into a list, and use the list.
The length of the file (number records) is dynamic, the width (elements in each record) is fixed. Which is the most efficient Pythonian way for this? I can change the file format if required (separation chars, etc).
New edit:
Based on some clues in the few replies I recieved, I managed to make progress, but the question is still there.
Here is what I have done, but at the end, I can not make it function like a 2-dim array, as can be seen in the attached code:

>>> test1 = open("C:/testdata1.txt", 'r') #This opens automatically as a 
#list, but with line breaks and no start and end brackets.
>>> test2 = test1.read()                # Convert to string
>>> test2 = test2.replace("\n","")      # Remove line breaks
>>> test2 = "[" + test2 + "]"           # Add brackets
>>> print(test2)

# The result looks like pure 2-dim list, but does not behave like one:
[[0x0,0x42,0x2A],[0x229,0x44,0x7C],[0x452,0x40,0x03],[0xCF9,0x4E,0x08], 
[0xF26,0x70,0x66],[0x1158,0x72,0xA6],[0x1388,0x72,0xB6],]

#This gives an error
>>> print(test2[1][2])
Traceback (most recent call last):
  File "<pyshell#79>", line 1, in <module>
    print(test2[1][2])
IndexError: string index out of range

#But it runs like one-dim array of chars
>>> print(test2[8])
4
>>>
# If I copy and paste the above list as a new list, it works nicely!

Can better use:
>>> with open("C:/testdata1.txt", 'r') as file:
     for line in file:
     file.read()
# But, again, reading result with line breaks, no brackets.
 '[0x229,0x44,0x7C],\n[0x452,0x40,0x03],\n[0xCF9,0x4E,0x08],
 \n[0xF26,0x70,0x66],\ n[0x1158,0x72,0xA6],\n[0x1388,0x72,0xB6],'

Upvotes: 0

Views: 150

Answers (3)

samtal
samtal

Reputation: 129

Well, after lots of tests and tries, I found the (simple) solution as follows:

I had an excel csv output file in the following format: (The commas and brackets were added in Excel):

[0x0,0x42,0x2A],
[0x229,0x44,0x7C],
[0x452,0x40,0x03],
[0xCF9,0x4E,0x08],

I manipulated the data and 'cleaned' it to make it Python 2-dim List format (had to remove line breaks, remove trailing comma, add start and end brackets, see my original question edit)):

[[0x0,0x42,0x2A],[0x229,0x44,0x7C],[0x452,0x40,0x03],[0xCF9,0x4E,0x08]]

The problem was that although it looks like 2-dim list, it was interpreted by Python as a string.

Eventually, with the help of an old thread 1, I found the solution on how to convert a 'legal' Python string into a Python List: myList = eval(string).

After that, the structure behaved like a nice 2-dim list (array). It was simple but took me hours to find.

The full code:

>>> test = eval("[" + (open("C:/testdata1.txt" , 'r').read().replace("\n","").rstrip(",") + "]"))
>>> print(hex(test[1][2]))
0x7c

Thanks to all who tried to help.

Upvotes: 0

whackamadoodle3000
whackamadoodle3000

Reputation: 6748

Try this:

with open("filename.txt","r") as f:
    [[hex(r) for r in e.split(",")] for e in f.read().replace("\n","")[1:-2].split("],[")]

Upvotes: 0

javidcf
javidcf

Reputation: 59691

If you really can format the file as you please, just make it so it is a Python module:

# bigarray.py

bigarray = [
[0xF26,0x70,0x66],[0x1158,0x72,0xA6],[0x1388,0x72,0xB6],
[0x15BA,0x4E,0x08],[0x17E9,0x70,0x92],[0x1A1D,0x72,0x94],
[0x1C4F,0x72,0xB4],[0x4409,0x4A,0x14], # etc. etc.
]

The from some other module:

# mymodule.py

from bigarray import bigarray

print(bigarray[1][2])

Upvotes: 2

Related Questions