Importing an irregular shaped array into Python

Question

I have some data generated in Mathematica that I need imported into Python. The way the data is generated relies on symbolic calculations so simply generating it in Python is out of the question. The data is an array of dimensions (126,2) but, where the first position in each element is an integer, the second position is a list of lists and the dimensions are not constant from element to element, for example:

`
    {
    {-9,{{4,2},{5,6},{8,10}}},
    {-2,{{3,6},{6,1}}}
    {4,{{3,6},{6,1},{3,6},{6,1},{3,6},{6,1},{3,6},{6,1}}}
    }
`

would be the first three elements. The second position in each element will always a list of 2-D lists. The goal here is to have this data imported as a numpy array such that I can call each element, no matter it's position.

I had some success with numpy.genfromtxt("data.txt",delimiters="}}}") which gives me the correct shape (126,2) but each element is simply "nan".

I had some more success with

`
with open("data.csv") as csvfile:
     reader = csv.reader(csvfile, delimiter=' ')
     for element in reader:
         print(np.asarray(element)[0])
`

This gives me the integer values as an array, which is great! For the second position in each element i tried:

`
def replace_all(text, dic):
    for i, j in dic.items():
        text = text.replace(i, j)
    return text
d={"{":"[","}":"]"}
with open("spinweights.csv") as csvfile:
     reader = csv.reader(csvfile, delimiter=',')
     it=0
     for element in reader:
         while it<1:
             curlToSq=replace_all(str(element[1]),d)
             print(np.asarray(curlToSq))
`

Where the replace_all function is changing all curly brackets in square brackets (the thinking here was this would make it easier to convert into a numpy array). The final line there does return an array...of shape () with none of it's objects subscriptable, which is what I need!

Any help is appreciated.

Answer 1

You may use Export with either "JSON" or "PythonExpression" to export the data to a file. Either of these can be directly read by Python.

Below I use ExportString to demonstrate.

With

x =
 {
  {-9, {{4, 2}, {5, 6}, {8, 10}}},
  {-2, {{3, 6}, {6, 1}}},
  {4, {{3, 6}, {6, 1}, {3, 6}, {6, 1}, {3, 6}, {6, 1}, {3, 6}, {6, 1}}}
  };

Then

ExportString[x, "JSON", "Compact" -> True]

[[-9,[[4,2],[5,6],[8,10]]],[-2,[[3,6],[6,1]]],[4,[[3,6],[6,1],[3,6],[6,1],[3,6],[6,1],[3,6],[6,1]]]]

or

ExportString[x, "PythonExpression"]

[[-9, [[4, 2], [5, 6], [8, 10]]], [-2, [[3, 6], [6, 1]]], [4, [[3, 6], [6, 1], [3, 6], [6, 1], [3, 6], [6, 1], [3, 6], [6, 1]]]]

Wolfram Language (a.k.a. Mathematica) can import and export many formats with the two above being some of the Basic Formats it supports. There is generally no need to custom code consumption of output as there is normally a format available that the other platform can naively (or has a library to) read.

Hope this helps.

Answer 2

I think converting this to a list structure will be easiest.

I added a , to the following:

In [22]: astr=""" { 
    ...:     {-9,{{4,2},{5,6},{8,10}}}, 
    ...:     {-2,{{3,6},{6,1}}}, 
    ...:     {4,{{3,6},{6,1},{3,6},{6,1},{3,6},{6,1},{3,6},{6,1}}} 
    ...:     }"""                                                               
In [23]: astr1=astr.replace('{','[').replace('}',']').replace('\n','')          
In [24]: astr1                                                                  
Out[24]: ' [    [-9,[[4,2],[5,6],[8,10]]],    [-2,[[3,6],[6,1]]],    [4,[[3,6],[6,1],[3,6],[6,1],[3,6],[6,1],[3,6],[6,1]]]    ]'

The are various string evaluators. eval is always available. ast is a bit safer. json.loads(astr1) also works.

In [25]: alist= eval(astr1)                                                     
In [26]: alist                                                                  
Out[26]: 
[[-9, [[4, 2], [5, 6], [8, 10]]],
 [-2, [[3, 6], [6, 1]]],
 [4, [[3, 6], [6, 1], [3, 6], [6, 1], [3, 6], [6, 1], [3, 6], [6, 1]]]]

If it must be an array, do:

In [27]: arr = np.array(alist)                                                  
In [28]: arr                                                                    
Out[28]: 
array([[-9, list([[4, 2], [5, 6], [8, 10]])],
       [-2, list([[3, 6], [6, 1]])],
       [4,
        list([[3, 6], [6, 1], [3, 6], [6, 1], [3, 6], [6, 1], [3, 6], [6, 1]])]],
      dtype=object)

This is (3,2). arr[:,0] is an array of integers, but arr[:,1] is an array of lists.

genfromtxt as a default tries to convert the input to floats (what it can convert becomes nan). It is intended for csv - a neat table of numbers of strings, with equal number of columns for each row.