CODEWITHSUNDEEP
pythonnumpy
Medo
Medo

Reputation: 1032

Numpy does not create binary file

When trying to write numpy matrix M to binary file as:

from io import open
X = [random.randint(0, 2 ** self.stages - 1)for _ in range(num)]
Matrix = np.asarray([list(map(int, list(x))) for x in X])

file_output = open('result.bin', 'wb')
M = np.ndarray(Matrix, dtype=np.float64)
file_output.write(M)
file_output.close()

I get this error:

Traceback (most recent call last):
  File "experiments.py", line 164, in <module>
    write_data(X, y)
  File "experiments.py", line 39, in write_data
    arr = np.ndarray(Matrix, dtype=np.float64)
ValueError: sequence too large; cannot be greater than 32

Can I know how to fix this? Thank you

Upvotes: 0

Views: 163

Answers (2)

hpaulj
hpaulj

Reputation: 231530

Replace:

M = np.ndarray(Matrix, dtype=np.float64)

with

M = Matrix.astype(np.float64)

np.array(Matrix, dtype=np.float64) would also work, but the astype is simpler.

I'm having some problems recreating your Matrix variable. What's its shape?

np.save is the best way of saving a multidimensional array to a file. There are other methods, but they don't save the shape and dtype information.

ndarray is wrong because the first (positional) argument is supposed to be the shape, not another array. The data if any is provided in the buffer parameter. ndarray is not normally used by beginners or even advanced numpy users.

What is Matrix supposed to be. When I try your code with a couple of parameters, I get an error in the map step:

In [495]: X = [np.random.randint(0, 2 ** 2 - 1)for _ in range(4)]
     ...: Matrix = np.asarray([list(map(int, list(x))) for x in X])
     ...: 
-----> 2 Matrix = np.asarray([list(map(int, list(x))) for x in X])
....
TypeError: 'int' object is not iterable
In [496]: X
Out[496]: [1, 2, 1, 0]

Is X just a list of numbers? Not some sort of list of arrays? And why the Matrix step? Is it trying to convert a nested list of lists into integers? Even though randint already creates integers? Then you follow with a conversion to float?

Upvotes: 1

JoshAdel
JoshAdel

Reputation: 68702

You can do this in one of two equivalent ways:

import numpy as np

a = np.random.normal(size=(10,10))
a.tofile('test1.dat')

with open('test2.dat', 'wb') as f:
    f.write(a.tobytes())

# diff test1.dat test2.dat

See the docs for tofile. However from the original example, it looks like Matrix is failing to be convert into an ndarray.

Upvotes: 1

Related Questions