CODEWITHSUNDEEP
pythonnumpy
Framester
Framester

Reputation: 35501

Numpy savez interprets my keys as filenames -> IOError

I am using numpy savez as recommended to save numpy arrays. As keys I use the names of the files I have loaded the data from. But it seems like savez is trying to use the filenames somehow. What should I do? I would like to avoid stripping the file names of their path and ending.

>>> import numpy
>>> arrs = {'data/a.text': numpy.array([1,2]),
            'data/b.text': numpy.array([3,4]),
            'data/c.text': numpy.array([5,6])}    
>>> numpy.savez('file.npz', **arrs)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/usr/lib/python2.6/dist-packages/numpy/lib/io.py", line 305, in savez
    fid = open(filename,'wb')
IOError: [Errno 2] No such file or directory: '/tmp/data/c.text.npy'

Upvotes: 3

Views: 1325

Answers (2)

jterrace
jterrace

Reputation: 67083

You can encode and decode the keys before you pass it to the savez function.

>>> import numpy
>>> import base64
>>> arrs = {'data/a.text': numpy.array([1,2]),
            'data/b.text': numpy.array([3,4]),
            'data/c.text': numpy.array([5,6])}
>>> numpy.savez('file.npz', **dict((base64.urlsafe_b64encode(k), v)
                                    for k,v in arrs.iteritems()))
>>> npzfile = numpy.load('file.npz')
>>> decoded = dict((base64.urlsafe_b64decode(k), v)
                   for k,v in npzfile.iteritems())
>>> decoded
{'data/c.text': array([5, 6]),
 'data/a.text': array([1, 2]),
 'data/b.text': array([3, 4])}

Upvotes: 2

Reinstate Monica
Reinstate Monica

Reputation: 4723

Probably savez builds temporary files with the names given in the dict. The filename has a / in it. When savez creates the file, it tries to use the given name and extension .npy (i.e. data/c.txt.py) as filename in the temp directory. However, the new path leads to a nonexisting subdirectory of temp, resulting in the error.

Solution would be: either replace the slash with something else, or maybe escape the filename.

(My earlier answer as too complicated and probably wrong.)

Upvotes: 1

Related Questions